author | Christian Urban <urbanc@in.tum.de> |
Thu, 23 Nov 2017 10:56:47 +0000 | |
changeset 153 | 4383809c176a |
parent 152 | 114a89518aea |
child 154 | 39c6b93718f0 |
permissions | -rw-r--r-- |
6 | 1 |
\documentclass{article} |
62 | 2 |
\usepackage{../style} |
78 | 3 |
\usepackage{../langs} |
153 | 4 |
\usepackage{tikz} |
5 |
\usepackage{pgf} |
|
6 |
\usepackage{pgfplots} |
|
7 |
||
8 |
\begin{filecontents}{re-python2.data} |
|
9 |
1 0.033 |
|
10 |
5 0.036 |
|
11 |
10 0.034 |
|
12 |
15 0.036 |
|
13 |
18 0.059 |
|
14 |
19 0.084 |
|
15 |
20 0.141 |
|
16 |
21 0.248 |
|
17 |
22 0.485 |
|
18 |
23 0.878 |
|
19 |
24 1.71 |
|
20 |
25 3.40 |
|
21 |
26 7.08 |
|
22 |
27 14.12 |
|
23 |
28 26.69 |
|
24 |
\end{filecontents} |
|
25 |
||
26 |
\begin{filecontents}{re-java.data} |
|
27 |
5 0.00298 |
|
28 |
10 0.00418 |
|
29 |
15 0.00996 |
|
30 |
16 0.01710 |
|
31 |
17 0.03492 |
|
32 |
18 0.03303 |
|
33 |
19 0.05084 |
|
34 |
20 0.10177 |
|
35 |
21 0.19960 |
|
36 |
22 0.41159 |
|
37 |
23 0.82234 |
|
38 |
24 1.70251 |
|
39 |
25 3.36112 |
|
40 |
26 6.63998 |
|
41 |
27 13.35120 |
|
42 |
28 29.81185 |
|
43 |
\end{filecontents} |
|
44 |
||
6 | 45 |
|
46 |
\begin{document} |
|
47 |
||
153 | 48 |
\section*{Coursework 8 (Scala, Regular Expressions, Brainf***)} |
6 | 49 |
|
79 | 50 |
This coursework is worth 10\%. It is about regular expressions, |
153 | 51 |
pattern matching and an interpreter. The first part is due on 30 |
152 | 52 |
November at 11pm; the second, more advanced part, is due on 21 |
153 | 53 |
December at 11pm. In the first part, you are asked to implement a |
54 |
regular expression matcher based on derivatives of regular |
|
55 |
expressions. The reason is that regular expression matching in Java |
|
56 |
can sometimes be extremely slow. The advanced part is about an |
|
57 |
interpreter for a very simple programming language.\bigskip |
|
62 | 58 |
|
59 |
\noindent |
|
152 | 60 |
\textbf{Important:} |
61 |
||
62 |
\begin{itemize} |
|
63 |
\item Make sure the files you submit can be processed by just calling\\ |
|
64 |
\mbox{\texttt{scala <<filename.scala>>}} on the commandline. Use the |
|
65 |
template files provided and do not make any changes to arguments of |
|
66 |
functions or to any types. You are free to implement any auxiliary |
|
67 |
function you might need. |
|
68 |
||
69 |
\item Do not use any mutable data structures in your |
|
70 |
submissions! They are not needed. This means you cannot create new |
|
71 |
\texttt{Array}s or \texttt{ListBuffer}s, for example. |
|
86 | 72 |
|
152 | 73 |
\item Do not use \texttt{return} in your code! It has a different |
74 |
meaning in Scala, than in Java. |
|
75 |
||
76 |
\item Do not use \texttt{var}! This declares a mutable variable. Only |
|
77 |
use \texttt{val}! |
|
78 |
||
79 |
\item Do not use any parallel collections! No \texttt{.par} therefore! |
|
80 |
Our testing and marking infrastructure is not set up for it. |
|
81 |
\end{itemize} |
|
82 |
||
83 |
\noindent |
|
84 |
Also note that the running time of each part will be restricted to a |
|
85 |
maximum of 360 seconds on my laptop |
|
6 | 86 |
|
87 |
||
110 | 88 |
\subsection*{Disclaimer} |
6 | 89 |
|
90 |
It should be understood that the work you submit represents |
|
68 | 91 |
your own effort! You have not copied from anyone else. An |
6 | 92 |
exception is the Scala code I showed during the lectures or |
93 |
uploaded to KEATS, which you can freely use.\bigskip |
|
94 |
||
95 |
||
68 | 96 |
\subsection*{Part 1 (6 Marks)} |
6 | 97 |
|
69 | 98 |
The task is to implement a regular expression matcher that is based on |
153 | 99 |
derivatives of regular expressions. Most of the functions are defined by |
100 |
recursion over regular expressions and can be elegantly implemented |
|
101 |
using Scala's pattern-matching. The implementation should deal with the |
|
102 |
following regular expressions, which have been predefined in the file |
|
103 |
\texttt{re.scala}: |
|
6 | 104 |
|
105 |
\begin{center} |
|
106 |
\begin{tabular}{lcll} |
|
107 |
$r$ & $::=$ & $\ZERO$ & cannot match anything\\ |
|
108 |
& $|$ & $\ONE$ & can only match the empty string\\ |
|
68 | 109 |
& $|$ & $c$ & can match a character (in this case $c$)\\ |
110 |
& $|$ & $r_1 + r_2$ & can match a string either with $r_1$ or with $r_2$\\ |
|
111 |
& $|$ & $r_1\cdot r_2$ & can match the first part of a string with $r_1$ and\\ |
|
112 |
& & & then the second part with $r_2$\\ |
|
6 | 113 |
& $|$ & $r^*$ & can match zero or more times $r$\\ |
114 |
\end{tabular} |
|
115 |
\end{center} |
|
116 |
||
68 | 117 |
\noindent |
152 | 118 |
Why? Knowing how to match regular expressions and strings will let you |
119 |
solve a lot of problems that vex other humans. Regular expressions are |
|
120 |
one of the fastest and simplest ways to match patterns in text, and |
|
121 |
are endlessly useful for searching, editing and analysing data in all |
|
122 |
sorts of places (for example analysing network traffic in order to |
|
123 |
detect security breaches). However, you need to be fast, otherwise you |
|
124 |
will stumble over problems such as recently reported at |
|
68 | 125 |
|
126 |
{\small |
|
127 |
\begin{itemize} |
|
128 |
\item[$\bullet$] \url{http://stackstatus.net/post/147710624694/outage-postmortem-july-20-2016} |
|
129 |
\item[$\bullet$] \url{https://vimeo.com/112065252} |
|
130 |
\item[$\bullet$] \url{http://davidvgalbraith.com/how-i-fixed-atom/} |
|
131 |
\end{itemize}} |
|
132 |
||
79 | 133 |
\subsubsection*{Tasks (file re.scala)} |
68 | 134 |
|
135 |
\begin{itemize} |
|
152 | 136 |
\item[(1a)] Implement a function, called \textit{nullable}, by |
137 |
recursion over regular expressions. This function tests whether a |
|
153 | 138 |
regular expression can match the empty string. This means given a |
152 | 139 |
regular expression it either returns true or false. |
6 | 140 |
|
141 |
\begin{center} |
|
142 |
\begin{tabular}{lcl} |
|
143 |
$\textit{nullable}(\ZERO)$ & $\dn$ & $\textit{false}$\\ |
|
144 |
$\textit{nullable}(\ONE)$ & $\dn$ & $\textit{true}$\\ |
|
145 |
$\textit{nullable}(c)$ & $\dn$ & $\textit{false}$\\ |
|
146 |
$\textit{nullable}(r_1 + r_2)$ & $\dn$ & $\textit{nullable}(r_1) \vee \textit{nullable}(r_2)$\\ |
|
147 |
$\textit{nullable}(r_1 \cdot r_2)$ & $\dn$ & $\textit{nullable}(r_1) \wedge \textit{nullable}(r_2)$\\ |
|
148 |
$\textit{nullable}(r^*)$ & $\dn$ & $\textit{true}$\\ |
|
149 |
\end{tabular} |
|
68 | 150 |
\end{center}\hfill[1 Mark] |
151 |
||
152 |
\item[(1b)] Implement a function, called \textit{der}, by recursion over |
|
153 |
regular expressions. It takes a character and a regular expression |
|
69 | 154 |
as arguments and calculates the derivative regular expression according |
155 |
to the rules: |
|
6 | 156 |
|
157 |
\begin{center} |
|
158 |
\begin{tabular}{lcl} |
|
159 |
$\textit{der}\;c\;(\ZERO)$ & $\dn$ & $\ZERO$\\ |
|
160 |
$\textit{der}\;c\;(\ONE)$ & $\dn$ & $\ZERO$\\ |
|
161 |
$\textit{der}\;c\;(d)$ & $\dn$ & $\textit{if}\; c = d\;\textit{then} \;\ONE \; \textit{else} \;\ZERO$\\ |
|
162 |
$\textit{der}\;c\;(r_1 + r_2)$ & $\dn$ & $(\textit{der}\;c\;r_1) + (\textit{der}\;c\;r_2)$\\ |
|
163 |
$\textit{der}\;c\;(r_1 \cdot r_2)$ & $\dn$ & $\textit{if}\;\textit{nullable}(r_1)$\\ |
|
164 |
& & $\textit{then}\;((\textit{der}\;c\;r_1)\cdot r_2) + (\textit{der}\;c\;r_2)$\\ |
|
165 |
& & $\textit{else}\;(\textit{der}\;c\;r_1)\cdot r_2$\\ |
|
166 |
$\textit{der}\;c\;(r^*)$ & $\dn$ & $(\textit{der}\;c\;r)\cdot (r^*)$\\ |
|
167 |
\end{tabular} |
|
69 | 168 |
\end{center} |
169 |
||
170 |
For example given the regular expression $r = (a \cdot b) \cdot c$, the derivatives |
|
171 |
w.r.t.~the characters $a$, $b$ and $c$ are |
|
172 |
||
173 |
\begin{center} |
|
174 |
\begin{tabular}{lcll} |
|
175 |
$\textit{der}\;a\;r$ & $=$ & $(\ONE \cdot b)\cdot c$ & ($= r'$)\\ |
|
176 |
$\textit{der}\;b\;r$ & $=$ & $(\ZERO \cdot b)\cdot c$\\ |
|
177 |
$\textit{der}\;c\;r$ & $=$ & $(\ZERO \cdot b)\cdot c$ |
|
178 |
\end{tabular} |
|
179 |
\end{center} |
|
180 |
||
181 |
Let $r'$ stand for the first derivative, then taking the derivatives of $r'$ |
|
182 |
w.r.t.~the characters $a$, $b$ and $c$ gives |
|
183 |
||
184 |
\begin{center} |
|
185 |
\begin{tabular}{lcll} |
|
186 |
$\textit{der}\;a\;r'$ & $=$ & $((\ZERO \cdot b) + \ZERO)\cdot c$ \\ |
|
187 |
$\textit{der}\;b\;r'$ & $=$ & $((\ZERO \cdot b) + \ONE)\cdot c$ & ($= r''$)\\ |
|
188 |
$\textit{der}\;c\;r'$ & $=$ & $((\ZERO \cdot b) + \ZERO)\cdot c$ |
|
189 |
\end{tabular} |
|
190 |
\end{center} |
|
191 |
||
192 |
One more example: Let $r''$ stand for the second derivative above, |
|
193 |
then taking the derivatives of $r''$ w.r.t.~the characters $a$, $b$ |
|
194 |
and $c$ gives |
|
195 |
||
196 |
\begin{center} |
|
197 |
\begin{tabular}{lcll} |
|
198 |
$\textit{der}\;a\;r''$ & $=$ & $((\ZERO \cdot b) + \ZERO) \cdot c + \ZERO$ \\ |
|
199 |
$\textit{der}\;b\;r''$ & $=$ & $((\ZERO \cdot b) + \ZERO) \cdot c + \ZERO$\\ |
|
152 | 200 |
$\textit{der}\;c\;r''$ & $=$ & $((\ZERO \cdot b) + \ZERO) \cdot c + \ONE$ & |
201 |
(is $\textit{nullable}$) |
|
69 | 202 |
\end{tabular} |
203 |
\end{center} |
|
204 |
||
205 |
Note, the last derivative can match the empty string, that is it is \textit{nullable}.\\ |
|
206 |
\mbox{}\hfill\mbox{[1 Mark]} |
|
6 | 207 |
|
68 | 208 |
\item[(1c)] Implement the function \textit{simp}, which recursively |
69 | 209 |
traverses a regular expression from the inside to the outside, and |
153 | 210 |
on the way simplifies every regular expression on the left (see |
211 |
below) to the regular expression on the right, except it does not |
|
212 |
simplify inside ${}^*$-regular expressions. |
|
6 | 213 |
|
68 | 214 |
\begin{center} |
69 | 215 |
\begin{tabular}{l@{\hspace{4mm}}c@{\hspace{4mm}}ll} |
6 | 216 |
$r \cdot \ZERO$ & $\mapsto$ & $\ZERO$\\ |
217 |
$\ZERO \cdot r$ & $\mapsto$ & $\ZERO$\\ |
|
218 |
$r \cdot \ONE$ & $\mapsto$ & $r$\\ |
|
219 |
$\ONE \cdot r$ & $\mapsto$ & $r$\\ |
|
220 |
$r + \ZERO$ & $\mapsto$ & $r$\\ |
|
221 |
$\ZERO + r$ & $\mapsto$ & $r$\\ |
|
222 |
$r + r$ & $\mapsto$ & $r$\\ |
|
223 |
\end{tabular} |
|
68 | 224 |
\end{center} |
225 |
||
69 | 226 |
For example the regular expression |
68 | 227 |
\[(r_1 + \ZERO) \cdot \ONE + ((\ONE + r_2) + r_3) \cdot (r_4 \cdot \ZERO)\] |
228 |
||
153 | 229 |
simplifies to just $r_1$. \textbf{Hint:} Regular expressions can be |
79 | 230 |
seen as trees and there are several methods for traversing |
153 | 231 |
trees. One of them corresponds to the inside-out traversal, which is |
232 |
sometimes also called post-order traversal. Furthermore, |
|
233 |
remember numerical expressions from school times: there you had expressions |
|
152 | 234 |
like $u + \ldots + (1 \cdot x) - \ldots (z + (y \cdot 0)) \ldots$ |
79 | 235 |
and simplification rules that looked very similar to rules |
236 |
above. You would simplify such numerical expressions by replacing |
|
237 |
for example the $y \cdot 0$ by $0$, or $1\cdot x$ by $x$, and then |
|
152 | 238 |
look whether more rules are applicable. If you organise the |
79 | 239 |
simplification in an inside-out fashion, it is always clear which |
153 | 240 |
rule should be applied next.\hfill[2 Marks] |
68 | 241 |
|
242 |
\item[(1d)] Implement two functions: The first, called \textit{ders}, |
|
69 | 243 |
takes a list of characters and a regular expression as arguments, and |
244 |
builds the derivative w.r.t.~the list as follows: |
|
68 | 245 |
|
246 |
\begin{center} |
|
247 |
\begin{tabular}{lcl} |
|
69 | 248 |
$\textit{ders}\;(Nil)\;r$ & $\dn$ & $r$\\ |
249 |
$\textit{ders}\;(c::cs)\;r$ & $\dn$ & |
|
68 | 250 |
$\textit{ders}\;cs\;(\textit{simp}(\textit{der}\;c\;r))$\\ |
251 |
\end{tabular} |
|
6 | 252 |
\end{center} |
253 |
||
78 | 254 |
Note that this function is different from \textit{der}, which only |
255 |
takes a single character. |
|
256 |
||
257 |
The second function, called \textit{matcher}, takes a string and a |
|
258 |
regular expression as arguments. It builds first the derivatives |
|
259 |
according to \textit{ders} and after that tests whether the resulting |
|
260 |
derivative regular expression can match the empty string (using |
|
261 |
\textit{nullable}). For example the \textit{matcher} will produce |
|
153 | 262 |
true for the regular expression $(a\cdot b)\cdot c$ and the string |
263 |
$abc$, but false if you give it the string $ab$. \hfill[1 Mark] |
|
78 | 264 |
|
153 | 265 |
\item[(1e)] Implement a function, called \textit{size}, by recursion |
78 | 266 |
over regular expressions. If a regular expression is seen as a tree, |
267 |
then \textit{size} should return the number of nodes in such a |
|
268 |
tree. Therefore this function is defined as follows: |
|
269 |
||
270 |
\begin{center} |
|
271 |
\begin{tabular}{lcl} |
|
272 |
$\textit{size}(\ZERO)$ & $\dn$ & $1$\\ |
|
273 |
$\textit{size}(\ONE)$ & $\dn$ & $1$\\ |
|
274 |
$\textit{size}(c)$ & $\dn$ & $1$\\ |
|
275 |
$\textit{size}(r_1 + r_2)$ & $\dn$ & $1 + \textit{size}(r_1) + \textit{size}(r_2)$\\ |
|
276 |
$\textit{size}(r_1 \cdot r_2)$ & $\dn$ & $1 + \textit{size}(r_1) + \textit{size}(r_2)$\\ |
|
277 |
$\textit{size}(r^*)$ & $\dn$ & $1 + \textit{size}(r)$\\ |
|
278 |
\end{tabular} |
|
279 |
\end{center} |
|
280 |
||
153 | 281 |
You can use \textit{size} in order to test how much the `evil' regular |
282 |
expression $(a^*)^* \cdot b$ grows when taking successive derivatives |
|
283 |
according the letter $a$ without simplification and then compare it to |
|
284 |
taking the derivative, but simplify the result. The sizes |
|
285 |
are given in \texttt{re.scala}. \hfill[1 Mark] |
|
286 |
\end{itemize} |
|
78 | 287 |
|
94
ae4708c851ee
updated
Christian Urban <christian dot urban at kcl dot ac dot uk>
parents:
86
diff
changeset
|
288 |
\subsection*{Background} |
78 | 289 |
|
94
ae4708c851ee
updated
Christian Urban <christian dot urban at kcl dot ac dot uk>
parents:
86
diff
changeset
|
290 |
Although easily implementable in Scala, the idea behind the derivative |
ae4708c851ee
updated
Christian Urban <christian dot urban at kcl dot ac dot uk>
parents:
86
diff
changeset
|
291 |
function might not so easy to be seen. To understand its purpose |
ae4708c851ee
updated
Christian Urban <christian dot urban at kcl dot ac dot uk>
parents:
86
diff
changeset
|
292 |
better, assume a regular expression $r$ can match strings of the form |
152 | 293 |
$c\!::\!cs$ (that means strings which start with a character $c$ and have |
153 | 294 |
some rest, or tail, $cs$). If you take the derivative of $r$ with |
295 |
respect to the character $c$, then you obtain a regular expression |
|
94
ae4708c851ee
updated
Christian Urban <christian dot urban at kcl dot ac dot uk>
parents:
86
diff
changeset
|
296 |
that can match all the strings $cs$. In other words, the regular |
153 | 297 |
expression $\textit{der}\;c\;r$ can match the same strings $c\!::\!cs$ |
94
ae4708c851ee
updated
Christian Urban <christian dot urban at kcl dot ac dot uk>
parents:
86
diff
changeset
|
298 |
that can be matched by $r$, except that the $c$ is chopped off. |
75 | 299 |
|
300 |
Assume now $r$ can match the string $abc$. If you take the derivative |
|
301 |
according to $a$ then you obtain a regular expression that can match |
|
302 |
$bc$ (it is $abc$ where the $a$ has been chopped off). If you now |
|
153 | 303 |
build the derivative $\textit{der}\;b\;(\textit{der}\;a\;r)$ you |
78 | 304 |
obtain a regular expression that can match the string $c$ (it is $bc$ |
305 |
where $b$ is chopped off). If you finally build the derivative of this |
|
306 |
according $c$, that is |
|
153 | 307 |
$\textit{der}\;c\;(\textit{der}\;b\;(\textit{der}\;a\;r))$, you obtain |
308 |
a regular expression that can match the empty string. You can test |
|
309 |
whether this is indeed the case using the function nullable, which is |
|
310 |
what your matcher is doing. |
|
75 | 311 |
|
153 | 312 |
The purpose of the $\textit{simp}$ function is to keep the regular |
313 |
expression small. Normally the derivative function makes the regular |
|
314 |
expression bigger (see the SEQ case and the example in (1b)) and the |
|
315 |
algorithm would be slower and slower over time. The $\textit{simp}$ |
|
316 |
function counters this increase in size and the result is that the |
|
317 |
algorithm is fast throughout. By the way, this algorithm is by Janusz |
|
318 |
Brzozowski who came up with the idea of derivatives in 1964 in his PhD |
|
319 |
thesis. |
|
75 | 320 |
|
78 | 321 |
\begin{center}\small |
322 |
\url{https://en.wikipedia.org/wiki/Janusz_Brzozowski_(computer_scientist)} |
|
323 |
\end{center} |
|
6 | 324 |
|
153 | 325 |
|
326 |
If you want to see how badly the regular expression matchers do in |
|
327 |
Java and Python with the `evil' regular expression, then have a look |
|
328 |
at the graphs below (you can try it out for yourself: have a look at |
|
329 |
the file \texttt{catastrophic.java} on KEATS). Compare this with the |
|
330 |
matcher you have implemented. How long can the string of $a$'s be |
|
331 |
in your matcher and stay within the 30 seconds time limit? |
|
332 |
||
333 |
\begin{center} |
|
334 |
\begin{tikzpicture} |
|
335 |
\begin{axis}[ |
|
336 |
title={Graph: $(a^*)^*\cdot b$ and strings |
|
337 |
$\underbrace{a\ldots a}_{n}$}, |
|
338 |
xlabel={$n$}, |
|
339 |
x label style={at={(1.05,0.0)}}, |
|
340 |
ylabel={time in secs}, |
|
341 |
enlargelimits=false, |
|
342 |
xtick={0,5,...,30}, |
|
343 |
xmax=33, |
|
344 |
ymax=35, |
|
345 |
ytick={0,5,...,30}, |
|
346 |
scaled ticks=false, |
|
347 |
axis lines=left, |
|
348 |
width=6cm, |
|
349 |
height=5.0cm, |
|
350 |
legend entries={Python, Java}, |
|
351 |
legend pos=outer north east] |
|
352 |
\addplot[blue,mark=*, mark options={fill=white}] table {re-python2.data}; |
|
353 |
\addplot[cyan,mark=*, mark options={fill=white}] table {re-java.data}; |
|
354 |
\end{axis} |
|
355 |
\end{tikzpicture} |
|
356 |
\end{center} |
|
357 |
\newpage |
|
358 |
||
359 |
\subsection*{Part 2 (4 Marks)} |
|
360 |
||
361 |
Comming from Java or C++, you might think Scala is a quite |
|
362 |
esotheric programming language. But remember, some serious companies |
|
363 |
have built their business on Scala. And there are far more esotheric |
|
364 |
languages out there. One is called \emph{brainf***}. Urban M\"uller |
|
365 |
developed this language in 1993. A close relative was already |
|
366 |
introduced in ... by Corado B\"ohm, an Italian computer pionier, who |
|
367 |
unfortunately died a few months ago. One feature of brainf*** is its |
|
368 |
minimalistic set of instructions. It has just 8 instructions, all of |
|
369 |
which are single characters. Despite this minimalism, this language, |
|
370 |
given enough memory, has been shown to be Turing complete. In this |
|
371 |
part you will implement an interpreter for this language. |
|
372 |
||
373 |
||
374 |
||
375 |
\subsubsection*{Tasks (file bf.scala)} |
|
376 |
||
377 |
\begin{itemize} |
|
378 |
\item[(2a)] |
|
379 |
||
380 |
\item[(2b)] |
|
381 |
||
382 |
\item[(2c)] |
|
383 |
||
384 |
\end{itemize}\bigskip |
|
385 |
||
386 |
||
387 |
||
388 |
||
6 | 389 |
\end{document} |
390 |
||
68 | 391 |
|
6 | 392 |
%%% Local Variables: |
393 |
%%% mode: latex |
|
394 |
%%% TeX-master: t |
|
395 |
%%% End: |