468
|
1 |
% Chapter Template
|
|
2 |
|
|
3 |
\chapter{Chapter Title Here} % Main chapter title
|
|
4 |
|
|
5 |
\label{ChapterX} % Change X to a consecutive number; for referencing this chapter elsewhere, use \ref{ChapterX}
|
|
6 |
|
500
|
7 |
|
|
8 |
|
|
9 |
|
468
|
10 |
%----------------------------------------------------------------------------------------
|
|
11 |
% SECTION 1
|
|
12 |
%----------------------------------------------------------------------------------------
|
|
13 |
|
|
14 |
\section{Properties of $\backslash c$}
|
|
15 |
|
|
16 |
To have a clear idea of what we can and
|
|
17 |
need to prove about the algorithms involving
|
|
18 |
Brzozowski's derivatives, there are a few
|
505
|
19 |
properties we need to be clear about .
|
468
|
20 |
\subsection{function $\backslash c$ is not 1-to-1}
|
|
21 |
\begin{center}
|
|
22 |
The derivative $w.r.t$ character $c$ is not one-to-one.
|
|
23 |
Formally,
|
|
24 |
$\exists r_1 \;r_2. r_1 \neq r_2 \mathit{and} r_1 \backslash c = r_2 \backslash c$
|
|
25 |
\end{center}
|
|
26 |
This property is trivially true for the
|
|
27 |
character regex example:
|
|
28 |
\begin{center}
|
|
29 |
$r_1 = e; \; r_2 = d;\; r_1 \backslash c = \ZERO = r_2 \backslash c$
|
|
30 |
\end{center}
|
|
31 |
But apart from the cases where the derivative
|
|
32 |
output is $\ZERO$, are there non-trivial results
|
|
33 |
of derivatives which contain strings?
|
|
34 |
The answer is yes.
|
|
35 |
For example,
|
|
36 |
\begin{center}
|
|
37 |
Let $r_1 = a^*b\;\quad r_2 = (a\cdot a^*)\cdot b + b$.\\
|
|
38 |
where $a$ is not nullable.\\
|
|
39 |
$r_1 \backslash c = ((a \backslash c)\cdot a^*)\cdot c + b \backslash c$\\
|
|
40 |
$r_2 \backslash c = ((a \backslash c)\cdot a^*)\cdot c + b \backslash c$
|
|
41 |
\end{center}
|
|
42 |
We start with two syntactically different regexes,
|
|
43 |
and end up with the same derivative result, which is
|
|
44 |
a "meaningful" regex because it contains strings.
|
|
45 |
We have rediscovered Arden's lemma:\\
|
|
46 |
\begin{center}
|
|
47 |
$A^*B = A\cdot A^* \cdot B + B$
|
|
48 |
\end{center}
|
|
49 |
|
|
50 |
|
|
51 |
%-----------------------------------
|
|
52 |
% SUBSECTION 1
|
|
53 |
%-----------------------------------
|
|
54 |
\subsection{Subsection 1}
|
505
|
55 |
To be completed.
|
500
|
56 |
|
|
57 |
|
|
58 |
|
|
59 |
|
|
60 |
|
|
61 |
|
|
62 |
|
468
|
63 |
%-----------------------------------
|
500
|
64 |
% SECTION 2
|
468
|
65 |
%-----------------------------------
|
|
66 |
|
500
|
67 |
\section{Finiteness Property}
|
|
68 |
In this section let us sketch our argument for why the size of the simplified
|
|
69 |
derivatives with the aggressive simplification function can be finitely bounded.
|
503
|
70 |
|
|
71 |
For convenience, we use a new datatype which we call $\rrexp$ to denote
|
500
|
72 |
the difference between it and annotated regular expressions.
|
503
|
73 |
\[ \rrexp ::= \RZERO \mid \RONE
|
|
74 |
\mid \RCHAR{c}
|
|
75 |
\mid \RSEQ{r_1}{r_2}
|
|
76 |
\mid \RALTS{rs}
|
|
77 |
\mid \RSTAR{r}
|
|
78 |
\]
|
|
79 |
For $\rrexp$ we throw away the bitcodes on the annotated regular expressions,
|
|
80 |
but keep everything else intact.
|
|
81 |
It is similar to annotated regular expressions being $\erase$-ed, but with all its structure preserved
|
500
|
82 |
(the $\erase$ function unfortunately does not preserve structure in the case of empty and singleton
|
503
|
83 |
$\ALTS$).
|
500
|
84 |
We denote the operation of erasing the bits and turning an annotated regular expression
|
503
|
85 |
into an $\rrexp{}$ as $\rerase{}$.
|
|
86 |
\begin{center}
|
|
87 |
\begin{tabular}{lcr}
|
|
88 |
$\rerase{\AZERO}$ & $=$ & $\RZERO$\\
|
|
89 |
$\rerase{\ASEQ{bs}{r_1}{r_2}}$ & $=$ & $\RSEQ{\rerase{r_1}}{\rerase{r_2}}$\\
|
|
90 |
$\rerase{\AALTS{bs}{rs}}$ & $ =$ & $ \RALTS{rs}$
|
|
91 |
\end{tabular}
|
|
92 |
\end{center}
|
|
93 |
%TODO: FINISH definition of rerase
|
|
94 |
Similarly we could define the derivative and simplification on
|
|
95 |
$\rrexp$, which would be identical to those we defined for plain $\rexp$s in chapter1,
|
|
96 |
except that now they can operate on alternatives taking multiple arguments.
|
505
|
97 |
|
|
98 |
\begin{center}
|
|
99 |
\begin{tabular}{lcr}
|
|
100 |
$\RALTS{rs} \backslash c$ & $=$ & $\RALTS{map\; (\_ \backslash c) \;rs}$\\
|
|
101 |
(other clauses omitted)
|
|
102 |
\end{tabular}
|
|
103 |
\end{center}
|
|
104 |
|
|
105 |
Now that $\rrexp$s do not have bitcodes on them, we can do the
|
|
106 |
duplicate removal without an equivalence relation:
|
|
107 |
\begin{center}
|
|
108 |
\begin{tabular}{lcl}
|
|
109 |
$\rdistinct{r :: rs}{rset}$ & $=$ & $\textit{if}(r \in \textit{rset}) \; \textit{then} \; \rdistinct{rs}{rset}$\\
|
|
110 |
& & $\textit{else}\; r::\rdistinct{rs}{(rset \cup \{r\})}$
|
|
111 |
\end{tabular}
|
|
112 |
\end{center}
|
|
113 |
%TODO: definition of rsimp (maybe only the alternative clause)
|
|
114 |
|
|
115 |
|
503
|
116 |
The reason why these definitions mirror precisely the corresponding operations
|
|
117 |
on annotated regualar expressions is that we can calculate sizes more easily:
|
|
118 |
|
|
119 |
\begin{lemma}
|
|
120 |
$\rsize{\rerase a} = \asize a$
|
|
121 |
\end{lemma}
|
|
122 |
A similar equation holds for annotated regular expressions' simplification
|
|
123 |
and the plain regular expressions' simplification:
|
|
124 |
\begin{lemma}
|
|
125 |
$\asize{\bsimp{a}} = \rsize{\rsimp{\rerase{a}}}$
|
|
126 |
\end{lemma}
|
|
127 |
Putting these two together we get a property that allows us to estimate the
|
|
128 |
size of an annotated regular expression derivative using its un-annotated counterpart:
|
|
129 |
\begin{lemma}
|
|
130 |
$\asize{\bderssimp{r}{s}} = \rsize{\rderssimp{\rerase{r}}{s}}$
|
|
131 |
\end{lemma}
|
|
132 |
Unless stated otherwise in this chapter all $\textit{rexp}$s without
|
|
133 |
bitcodes are seen as $\rrexp$s.
|
505
|
134 |
We also use $r_1 + r_2$ and $\RALTS{[r_1, r_2]}$ interchageably
|
|
135 |
as the former suits people's intuitive way of stating a binary alternative
|
|
136 |
regular expression.
|
500
|
137 |
|
|
138 |
Suppose
|
|
139 |
we have a size function for bitcoded regular expressions, written
|
|
140 |
$\llbracket r\rrbracket$, which counts the number of nodes if we regard $r$ as a tree
|
|
141 |
(we omit the precise definition; ditto for lists $\llbracket r\!s\rrbracket$). For this we show that for every $r$
|
|
142 |
there exists a bound $N$
|
|
143 |
such that
|
|
144 |
|
|
145 |
\begin{center}
|
503
|
146 |
$\forall s. \; \llbracket{\bderssimp{r}{s}}\rrbracket \leq N$
|
500
|
147 |
\end{center}
|
|
148 |
|
|
149 |
\noindent
|
|
150 |
We prove this by induction on $r$. The base cases for $\AZERO$,
|
|
151 |
$\AONE \textit{bs}$ and $\ACHAR \textit{bs} c$ are straightforward. The interesting case is
|
503
|
152 |
for sequences of the form $\ASEQ{bs}{r_1}{r_2}$. In this case our induction
|
|
153 |
hypotheses state $\exists N_1. \forall s. \; \llbracket \bderssimp{r}{s} \rrbracket \leq N_1$ and
|
|
154 |
$\exists N_2. \forall s. \; \llbracket \bderssimp{r_2}{s} \rrbracket \leq N_2$. We can reason as follows
|
500
|
155 |
%
|
|
156 |
\begin{center}
|
|
157 |
\begin{tabular}{lcll}
|
503
|
158 |
& & $ \llbracket \bderssimp{\ASEQ{bs}{r_1}{r_2} }{s} \rrbracket$\\
|
|
159 |
& $ = $ & $\llbracket bsimp\,(\textit{ALTs}\;bs\;(\ASEQ{nil}{\bderssimp{ r_1}{s}}{ r_2} ::
|
|
160 |
[\bderssimp{ r_2}{s'} \;|\; s' \in \textit{Suffix}( r_1, s)]))\rrbracket $ & (1) \\
|
500
|
161 |
& $\leq$ &
|
503
|
162 |
$\llbracket\textit{\distinctWith}\,((\ASEQ{nil}{\bderssimp{r_1}{s}}{r_2}$) ::
|
|
163 |
[$\bderssimp{ r_2}{s'} \;|\; s' \in \textit{Suffix}( r_1, s)])\,\approx\;{}\rrbracket + 1 $ & (2) \\
|
|
164 |
& $\leq$ & $\llbracket\ASEQ{bs}{\bderssimp{ r_1}{s}}{r_2}\rrbracket +
|
|
165 |
\llbracket\textit{distinctWith}\,[\bderssimp{r_2}{s'} \;|\; s' \in \textit{Suffix}( r_1, s)]\,\approx\;{}\rrbracket + 1 $ & (3) \\
|
500
|
166 |
& $\leq$ & $N_1 + \llbracket r_2\rrbracket + 2 +
|
503
|
167 |
\llbracket \distinctWith\,[ \bderssimp{r_2}{s'} \;|\; s' \in \textit{Suffix}( r_1, s)] \,\approx\;\rrbracket$ & (4)\\
|
500
|
168 |
& $\leq$ & $N_1 + \llbracket r_2\rrbracket + 2 + l_{N_{2}} * N_{2}$ & (5)
|
|
169 |
\end{tabular}
|
|
170 |
\end{center}
|
|
171 |
|
|
172 |
% tell Chengsong about Indian paper of closed forms of derivatives
|
|
173 |
|
|
174 |
\noindent
|
503
|
175 |
where in (1) the $\textit{Suffix}( r_1, s)$ are the all the suffixes of $s$ where $\bderssimp{ r_1}{s'}$ is nullable ($s'$ being a suffix of $s$).
|
500
|
176 |
The reason why we could write the derivatives of a sequence this way is described in section 2.
|
503
|
177 |
The term (2) is used to control (1).
|
|
178 |
That is because one can obtain an overall
|
500
|
179 |
smaller regex list
|
|
180 |
by flattening it and removing $\ZERO$s first before applying $\distinctWith$ on it.
|
|
181 |
Section 3 is dedicated to its proof.
|
503
|
182 |
In (3) we know that $\llbracket \ASEQ{bs}{(\bderssimp{ r_1}{s}}{r_2}\rrbracket$ is
|
500
|
183 |
bounded by $N_1 + \llbracket{}r_2\rrbracket + 1$. In (5) we know the list comprehension contains only regular expressions of size smaller
|
|
184 |
than $N_2$. The list length after $\distinctWith$ is bounded by a number, which we call $l_{N_2}$. It stands
|
|
185 |
for the number of distinct regular expressions smaller than $N_2$ (there can only be finitely many of them).
|
|
186 |
We reason similarly for $\STAR$.\medskip
|
|
187 |
|
|
188 |
\noindent
|
|
189 |
Clearly we give in this finiteness argument (Step (5)) a very loose bound that is
|
|
190 |
far from the actual bound we can expect. We can do better than this, but this does not improve
|
|
191 |
the finiteness property we are proving. If we are interested in a polynomial bound,
|
|
192 |
one would hope to obtain a similar tight bound as for partial
|
|
193 |
derivatives introduced by Antimirov \cite{Antimirov95}. After all the idea with
|
|
194 |
$\distinctWith$ is to maintain a ``set'' of alternatives (like the sets in
|
|
195 |
partial derivatives). Unfortunately to obtain the exact same bound would mean
|
|
196 |
we need to introduce simplifications, such as
|
|
197 |
$(r_1 + r_2) \cdot r_3 \longrightarrow (r_1 \cdot r_3) + (r_2 \cdot r_3)$,
|
|
198 |
which exist for partial derivatives. However, if we introduce them in our
|
|
199 |
setting we would lose the POSIX property of our calculated values. We leave better
|
|
200 |
bounds for future work.\\[-6.5mm]
|
|
201 |
|
468
|
202 |
|
507
|
203 |
|
|
204 |
%-----------------------------------
|
|
205 |
% SECTION ?
|
|
206 |
%-----------------------------------
|
|
207 |
\section{preparatory properties for getting a finiteness bound}
|
|
208 |
Before we get to the proof that says the intermediate result of our lexer will
|
|
209 |
remain finitely bounded, which is an important efficiency/liveness guarantee,
|
|
210 |
we shall first develop a few preparatory properties and definitions to
|
|
211 |
make the process of proving that a breeze.
|
|
212 |
|
|
213 |
We define rewriting relations for $\rrexp$s, which allows us to do the
|
|
214 |
same trick as we did for the correctness proof,
|
|
215 |
but this time we will have stronger equalities established.
|
|
216 |
\subsection{"hrewrite" relation}
|
514
|
217 |
List of 1-step rewrite rules for regular expressions simplification without bitcodes:
|
515
|
218 |
\begin{center}
|
|
219 |
\begin{tabular}{lcl}
|
|
220 |
$r_1 \cdot \ZERO$ & $\rightarrow$ & $\ZERO$
|
|
221 |
\end{tabular}
|
|
222 |
\end{center}
|
|
223 |
|
|
224 |
And we define an "grewrite" relation that works on lists:
|
|
225 |
\begin{center}
|
|
226 |
\begin{tabular}{lcl}
|
|
227 |
$ \ZERO :: rs$ & $\rightarrow_g$ & $rs$
|
|
228 |
\end{tabular}
|
|
229 |
\end{center}
|
507
|
230 |
|
|
231 |
|
|
232 |
|
515
|
233 |
With these relations we prove
|
|
234 |
\begin{lemma}
|
|
235 |
$rs \rightarrow rs' \implies \RALTS{rs} \rightarrow \RALTS{rs'}$
|
|
236 |
\end{lemma}
|
|
237 |
which enables us to prove "closed forms" equalities such as
|
|
238 |
\begin{lemma}
|
|
239 |
$\sflat{(r_1 \cdot r_2) \backslash s} = \RALTS{ (r_1 \backslash s) \cdot r_2 :: (\map (r_2 \backslash \_) (\vsuf{s}{r_1}))}$
|
|
240 |
\end{lemma}
|
|
241 |
|
|
242 |
But the most involved part of the above lemma is proving the following:
|
|
243 |
\begin{lemma}
|
|
244 |
$\bsimp{\RALTS{rs @ \RALTS{rs_1} @ rs'}} = \bsimp{\RALTS{rs @rs_1 @ rs'}} $
|
|
245 |
\end{lemma}
|
|
246 |
which is needed in proving things like
|
|
247 |
\begin{lemma}
|
|
248 |
$r \rightarrow_f r' \implies \rsimp{r} \rightarrow \rsimp{r'}$
|
|
249 |
\end{lemma}
|
|
250 |
.
|
|
251 |
Fortunately this is provable by induction on the list $rs$
|
507
|
252 |
|
468
|
253 |
%----------------------------------------------------------------------------------------
|
507
|
254 |
% SECTION ??
|
468
|
255 |
%----------------------------------------------------------------------------------------
|
|
256 |
|
500
|
257 |
\section{"Closed Forms" of regular expressions' derivatives w.r.t strings}
|
503
|
258 |
To embark on getting the "closed forms" of regexes, we first
|
|
259 |
define a few auxiliary definitions to make expressing them smoothly.
|
|
260 |
|
|
261 |
\begin{center}
|
|
262 |
\begin{tabular}{ccc}
|
|
263 |
$\sflataux{\AALTS{ }{r :: rs}}$ & $=$ & $\sflataux{r} @ rs$\\
|
|
264 |
$\sflataux{\AALTS{ }{[]}}$ & $ = $ & $ []$\\
|
|
265 |
$\sflataux r$ & $=$ & $ [r]$
|
|
266 |
\end{tabular}
|
|
267 |
\end{center}
|
|
268 |
The intuition behind $\sflataux{\_}$ is to break up nested regexes
|
|
269 |
of the $(\ldots((r_1 + r_2) + r_3) + \ldots )$(left-associated) shape
|
|
270 |
into a more "balanced" list: $\AALTS{\_}{[r_1,\, r_2 ,\, r_3, \ldots]}$.
|
|
271 |
It will return the singleton list $[r]$ otherwise.
|
|
272 |
|
|
273 |
$\sflat{\_}$ works the same as $\sflataux{\_}$, except that it keeps
|
|
274 |
the output type a regular expression, not a list:
|
|
275 |
\begin{center}
|
|
276 |
\begin{tabular}{ccc}
|
|
277 |
$\sflat{\AALTS{ }{r :: rs}}$ & $=$ & $\sflataux{r} @ rs$\\
|
|
278 |
$\sflat{\AALTS{ }{[]}}$ & $ = $ & $ \AALTS{ }{[]}$\\
|
|
279 |
$\sflat r$ & $=$ & $ [r]$
|
|
280 |
\end{tabular}
|
|
281 |
\end{center}
|
|
282 |
$\sflataux{\_}$ and $\sflat{\_}$ is only recursive in terms of the
|
|
283 |
first element of the list of children of
|
|
284 |
an alternative regex ($\AALTS{ }{rs}$), and is purposefully built for dealing with the regular expression
|
|
285 |
$r_1 \cdot r_2 \backslash s$.
|
|
286 |
|
|
287 |
With $\sflat{\_}$ and $\sflataux{\_}$ we make
|
|
288 |
precise what "closed forms" we have for the sequence derivatives and their simplifications,
|
|
289 |
in other words, how can we construct $(r_1 \cdot r_2) \backslash s$
|
|
290 |
and $\bderssimp{(r_1\cdot r_2)}{s}$,
|
|
291 |
if we are only allowed to use a combination of $r_1 \backslash s'$ ($\bderssimp{r_1}{s'}$)
|
|
292 |
and $r_2 \backslash s'$ ($\bderssimp{r_2}{s'}$), where $s'$ ranges over
|
|
293 |
the substring of $s$?
|
|
294 |
First let's look at a series of derivatives steps on a sequence
|
|
295 |
regular expression, (assuming) that each time the first
|
|
296 |
component of the sequence is always nullable):
|
|
297 |
\begin{center}
|
|
298 |
|
|
299 |
$r_1 \cdot r_2 \quad \longrightarrow_{\backslash c} \quad r_1 \backslash c \cdot r_2 + r_2 \backslash c \quad \longrightarrow_{\backslash c'} \quad (r_1 \backslash cc' \cdot r_2 + r_2 \backslash c') + r_2 \backslash cc' \longrightarrow_{\backslash c''} \quad$\\
|
|
300 |
$((r_1 \backslash cc'c'' \cdot r_2 + r_2 \backslash c'') + r_2 \backslash c'c'') + r_2 \backslash cc'c'' \longrightarrow_{\backslash c''} \quad
|
|
301 |
\ldots$
|
|
302 |
|
|
303 |
\end{center}
|
|
304 |
%TODO: cite indian paper
|
|
305 |
Indianpaper have come up with a slightly more formal way of putting the above process:
|
|
306 |
\begin{center}
|
|
307 |
$r_1 \cdot r_2 \backslash (c_1 :: c_2 ::\ldots c_n) \myequiv r_1 \backslash (c_1 :: c_2:: \ldots c_n) +
|
|
308 |
\delta(\nullable(r_1 \backslash (c_1 :: c_2 \ldots c_{n-1}) ), r_2 \backslash (c_n)) + \ldots
|
|
309 |
+ \delta (\nullable(r_1), r_2\backslash (c_1 :: c_2 ::\ldots c_n))$
|
|
310 |
\end{center}
|
|
311 |
where $\delta(b, r)$ will produce $r$
|
|
312 |
if $b$ evaluates to true,
|
|
313 |
and $\ZERO$ otherwise.
|
|
314 |
|
|
315 |
But the $\myequiv$ sign in the above equation means language equivalence instead of syntactical
|
|
316 |
equivalence. To make this intuition useful
|
|
317 |
for a formal proof, we need something
|
|
318 |
stronger than language equivalence.
|
|
319 |
With the help of $\sflat{\_}$ we can state the equation in Indianpaper
|
|
320 |
more rigorously:
|
500
|
321 |
\begin{lemma}
|
505
|
322 |
$\sflat{(r_1 \cdot r_2) \backslash s} = \RALTS{ (r_1 \backslash s) \cdot r_2 :: (\map (r_2 \backslash \_) (\vsuf{s}{r_1}))}$
|
500
|
323 |
\end{lemma}
|
505
|
324 |
|
|
325 |
The function $\vsuf{\_}{\_}$ is defined recursively on the structure of the string:
|
|
326 |
|
503
|
327 |
\begin{center}
|
|
328 |
\begin{tabular}{lcl}
|
|
329 |
$\vsuf{[]}{\_} $ & $=$ & $[]$\\
|
505
|
330 |
$\vsuf{c::cs}{r_1}$ & $ =$ & $ \textit{if} (\rnullable{r_1}) \textit{then} \; (\vsuf{cs}{(\rder{c}{r_1})}) @ [c :: cs]$\\
|
|
331 |
&& $\textit{else} \; (\vsuf{cs}{(\rder{c}{r_1}) }) $
|
503
|
332 |
\end{tabular}
|
|
333 |
\end{center}
|
|
334 |
It takes into account which prefixes $s'$ of $s$ would make $r \backslash s'$ nullable,
|
|
335 |
and keep a list of suffixes $s''$ such that $s' @ s'' = s$. The list is sorted in
|
|
336 |
the order $r_2\backslash s''$ appears in $(r_1\cdot r_2)\backslash s$.
|
505
|
337 |
In essence, $\vsuf{\_}{\_}$ is doing a "virtual derivative" of $r_1 \cdot r_2$, but instead of producing
|
503
|
338 |
the entire result of $(r_1 \cdot r_2) \backslash s$,
|
505
|
339 |
it only stores all the $s''$ such that $r_2 \backslash s''$ are occurring terms in $(r_1\cdot r_2)\backslash s$.
|
500
|
340 |
|
503
|
341 |
With this we can also add simplifications to both sides and get
|
|
342 |
\begin{lemma}
|
|
343 |
$\rsimp{\sflat{(r_1 \cdot r_2) \backslash s} }= \rsimp{\AALTS{[[]}{ (r_1 \backslash s) \cdot r_2 :: (\map (r_2 \backslash \_) (\vsuf{s}{r_1}))}}$
|
|
344 |
\end{lemma}
|
|
345 |
Together with the idempotency property of $\rsimp$,
|
|
346 |
%TODO: state the idempotency property of rsimp
|
|
347 |
\begin{lemma}
|
505
|
348 |
$\rsimp(r) = \rsimp (\rsimp(r))$
|
503
|
349 |
\end{lemma}
|
505
|
350 |
it is possible to convert the above lemma to obtain a "closed form"
|
|
351 |
for derivatives nested with simplification:
|
|
352 |
\begin{lemma}
|
|
353 |
$\rderssimp{(r_1 \cdot r_2)}{s} = \rsimp{\AALTS{[[]}{ (r_1 \backslash s) \cdot r_2 :: (\map (r_2 \backslash \_) (\vsuf{s}{r_1}))}}$
|
|
354 |
\end{lemma}
|
|
355 |
And now the reason we have (1) in section 1 is clear:
|
|
356 |
$\rsize{\rsimp{\RALTS{ (r_1 \backslash s) \cdot r_2 :: (\map \;(r_2 \backslash \_) \; (\vsuf{s}{r_1}))}}}$,
|
|
357 |
is equal to $\rsize{\rsimp{\RALTS{ ((r_1 \backslash s) \cdot r_2 :: (\map \; (r_2 \backslash \_) \; (\textit{Suffix}(r1, s))))}}}$
|
|
358 |
as $\vsuf{s}{r_1}$ is one way of expressing the list $\textit{Suffix}( r_1, s)$.
|
500
|
359 |
|
|
360 |
%----------------------------------------------------------------------------------------
|
|
361 |
% SECTION 3
|
|
362 |
%----------------------------------------------------------------------------------------
|
|
363 |
|
|
364 |
\section{interaction between $\distinctWith$ and $\flts$}
|
|
365 |
Note that it is not immediately obvious that
|
505
|
366 |
\begin{center}
|
|
367 |
$\llbracket \distinctWith (\flts \textit{rs}) = \phi \rrbracket \leq \llbracket \distinctWith \textit{rs} = \phi \rrbracket $.
|
|
368 |
\end{center}
|
|
369 |
The intuition is that if we remove duplicates from the $\textit{LHS}$, at least the same amount of
|
|
370 |
duplicates will be removed from the list $\textit{rs}$ in the $\textit{RHS}$.
|
|
371 |
|
|
372 |
%----------------------------------------------------------------------------------------
|
|
373 |
% SECTION 4
|
|
374 |
%----------------------------------------------------------------------------------------
|
|
375 |
\section{a bound for the star regular expression}
|
|
376 |
We have shown how to control the size of the sequence regular expression $r_1\cdot r_2$ using
|
|
377 |
the "closed form" of $(r_1 \cdot r_2) \backslash s$ and then
|
|
378 |
the property of the $\distinct$ function.
|
|
379 |
Now we try to get a bound on $r^* \backslash s$ as well.
|
|
380 |
Again, we first look at how a star's derivatives evolve, if they grow maximally:
|
|
381 |
\begin{center}
|
|
382 |
|
|
383 |
$r^* \quad \longrightarrow_{\backslash c} \quad (r\backslash c) \cdot r^* \quad \longrightarrow_{\backslash c'} \quad
|
|
384 |
r \backslash cc' \cdot r^* + r \backslash c' \cdot r^* \quad \longrightarrow_{\backslash c''} \quad
|
|
385 |
(r_1 \backslash cc'c'' \cdot r^* + r \backslash c'') + (r \backslash c'c'' \cdot r^* + r \backslash c'' \cdot r^*) \quad \longrightarrow_{\backslash c'''}
|
|
386 |
\quad \ldots$
|
|
387 |
|
|
388 |
\end{center}
|
|
389 |
When we have a string $s = c :: c' :: c'' \ldots$ such that $r \backslash c$, $r \backslash cc'$, $r \backslash c'$,
|
|
390 |
$r \backslash cc'c''$, $r \backslash c'c''$, $r\backslash c''$ etc. are all nullable,
|
|
391 |
the number of terms in $r^* \backslash s$ will grow exponentially, causing the size
|
|
392 |
of the derivatives $r^* \backslash s$ to grow exponentially, even if we do not
|
|
393 |
count the possible size explosions of $r \backslash c$ themselves.
|
468
|
394 |
|
505
|
395 |
Thanks to $\flts$ and $\distinctWith$, we are able to open up regexes like
|
|
396 |
$(r_1 \backslash cc'c'' \cdot r^* + r \backslash c'') + (r \backslash c'c'' \cdot r^* + r \backslash c'' \cdot r^*) $
|
|
397 |
into $\RALTS{[r_1 \backslash cc'c'' \cdot r^*, r \backslash c'', r \backslash c'c'' \cdot r^*, r \backslash c'' \cdot r^*]}$
|
|
398 |
and then de-duplicate terms of the form $r\backslash s' \cdot r^*$ ($s'$ being a substring of $s$).
|
|
399 |
For this we define $\hflataux{\_}$ and $\hflat{\_}$, similar to $\sflataux{\_}$ and $\sflat{\_}$:
|
|
400 |
%TODO: definitions of and \hflataux \hflat
|
|
401 |
\begin{center}
|
|
402 |
\begin{tabular}{ccc}
|
|
403 |
$\hflataux{r_1 + r_2}$ & $=$ & $\hflataux{r_1} @ \hflataux{r_2}$\\
|
|
404 |
$\hflataux r$ & $=$ & $ [r]$
|
|
405 |
\end{tabular}
|
|
406 |
\end{center}
|
|
407 |
|
|
408 |
\begin{center}
|
|
409 |
\begin{tabular}{ccc}
|
|
410 |
$\hflat{r_1 + r_2}$ & $=$ & $\RALTS{\hflataux{r_1} @ \hflataux{r_2}}$\\
|
|
411 |
$\hflat r$ & $=$ & $ r$
|
|
412 |
\end{tabular}
|
|
413 |
\end{center}
|
|
414 |
Again these definitions are tailor-made for dealing with alternatives that have
|
|
415 |
originated from a star's derivatives, so we don't attempt to open up all possible
|
|
416 |
regexes of the form $\RALTS{rs}$, where $\textit{rs}$ might not contain precisely 2
|
|
417 |
elements.
|
|
418 |
We give a predicate for such "star-created" regular expressions:
|
|
419 |
\begin{center}
|
|
420 |
\begin{tabular}{lcr}
|
506
|
421 |
& & $\createdByStar{(\RSEQ{ra}{\RSTAR{rb}}) }$\\
|
505
|
422 |
$\createdByStar{r_1} \land \createdByStar{r_2} $ & $ \Longrightarrow$ & $\createdByStar{(r_1 + r_2)}$
|
|
423 |
\end{tabular}
|
|
424 |
\end{center}
|
|
425 |
|
506
|
426 |
These definitions allows us the flexibility to talk about
|
|
427 |
regular expressions in their most convenient format,
|
|
428 |
for example, flattened out $\RALTS{[r_1, r_2, \ldots, r_n]} $
|
|
429 |
instead of binary-nested: $((r_1 + r_2) + (r_3 + r_4)) + \ldots$.
|
|
430 |
These definitions help express that certain classes of syntatically
|
|
431 |
distinct regular expressions are actually the same under simplification.
|
|
432 |
This is not entirely true for annotated regular expressions:
|
|
433 |
%TODO: bsimp bders \neq bderssimp
|
|
434 |
\begin{center}
|
|
435 |
$(1+ (c\cdot \ASEQ{bs}{c^*}{c} ))$
|
|
436 |
\end{center}
|
|
437 |
For bit-codes, the order in which simplification is applied
|
|
438 |
might cause a difference in the location they are placed.
|
|
439 |
If we want something like
|
|
440 |
\begin{center}
|
|
441 |
$\bderssimp{r}{s} \myequiv \bsimp{\bders{r}{s}}$
|
|
442 |
\end{center}
|
|
443 |
Some "canonicalization" procedure is required,
|
|
444 |
which either pushes all the common bitcodes to nodes
|
|
445 |
as senior as possible:
|
|
446 |
\begin{center}
|
|
447 |
$_{bs}(_{bs_1 @ bs'}r_1 + _{bs_1 @ bs''}r_2) \rightarrow _{bs @ bs_1}(_{bs'}r_1 + _{bs''}r_2) $
|
|
448 |
\end{center}
|
|
449 |
or does the reverse. However bitcodes are not of interest if we are talking about
|
|
450 |
the $\llbracket r \rrbracket$ size of a regex.
|
|
451 |
Therefore for the ease and simplicity of producing a
|
|
452 |
proof for a size bound, we are happy to restrict ourselves to
|
|
453 |
unannotated regular expressions, and obtain such equalities as
|
|
454 |
\begin{lemma}
|
|
455 |
$\rsimp{(r_1 + r_2)} = \rsimp{\RALTS{\hflataux{r_1} @ \hflataux{r_2}}}$
|
|
456 |
\end{lemma}
|
|
457 |
|
|
458 |
\begin{proof}
|
|
459 |
By using the rewriting relation $\rightsquigarrow$
|
|
460 |
\end{proof}
|
|
461 |
%TODO: rsimp sflat
|
|
462 |
And from this we obtain a proof that a star's derivative will be the same
|
|
463 |
as if it had all its nested alternatives created during deriving being flattened out:
|
|
464 |
For example,
|
|
465 |
\begin{lemma}
|
|
466 |
$\createdByStar{r} \implies \rsimp{r} = \rsimp{\RALTS{\hflataux{r}}}$
|
|
467 |
\end{lemma}
|
|
468 |
\begin{proof}
|
|
469 |
By structural induction on $r$, where the induction rules are these of $\createdByStar{_}$.
|
|
470 |
\end{proof}
|
|
471 |
% The simplification of a flattened out regular expression, provided it comes
|
|
472 |
%from the derivative of a star, is the same as the one nested.
|
|
473 |
|
|
474 |
|
|
475 |
|
|
476 |
|
|
477 |
|
|
478 |
|
|
479 |
|
|
480 |
|
|
481 |
|
505
|
482 |
One might wonder the actual bound rather than the loose bound we gave
|
|
483 |
for the convenience of a easier proof.
|
|
484 |
How much can the regex $r^* \backslash s$ grow? As hinted earlier,
|
|
485 |
they can grow at a maximum speed
|
506
|
486 |
exponential $w.r.t$ the number of characters,
|
|
487 |
but will eventually level off when the string $s$ is long enough,
|
505
|
488 |
as suggested by the finiteness bound proof.
|
506
|
489 |
And unfortunately we have concrete examples
|
|
490 |
where such regexes grew exponentially large before levelling off:
|
516
|
491 |
$(a ^ * + (aa) ^ * + (aaa) ^ * + \ldots +
|
|
492 |
(\underbrace{a \ldots a}_{\text{n a's}})^*$ will already have a maximum
|
506
|
493 |
size that is exponential on the number $n$.
|
505
|
494 |
%TODO: give out a graph showing how the size of the regex grows and levels off at a size exponential to the original regex
|
|
495 |
|
|
496 |
|
506
|
497 |
While the tight upper bound will definitely be a lot lower than
|
|
498 |
the one we gave for the finiteness proof,
|
|
499 |
it will still be at least expoential, under our current simplification rules.
|
|
500 |
|
|
501 |
This suggests stronger simplifications that keep the tight bound polynomial.
|
|
502 |
|
|
503 |
%----------------------------------------------------------------------------------------
|
|
504 |
% SECTION 5
|
|
505 |
%----------------------------------------------------------------------------------------
|
516
|
506 |
\section{A Stronger Version of Simplification}
|
506
|
507 |
%TODO: search for isabelle proofs of algorithms that check equivalence
|
|
508 |
|
|
509 |
|