\documentclass{article}
\usepackage{../style}
\usepackage{../langs}

\begin{document}

\section*{August Exam (Scala):  Chat Log Mining}

This coursework is worth 50\%. It is about mining a log of an online
chat between 85 participants. The log is given as a csv-list in the file
\texttt{log.csv}. The log is an unordered list containing information which
message has been sent, by whom, when and in response to which other
message. Each message has also a number and a unique hash code.\bigskip

\noindent 
\textbf{Important:} Make sure the file you submit can be processed 
by just calling

\begin{center}
  \texttt{scala <<filename.scala>>}
\end{center}

\noindent
Do not use any mutable data structures in your
submission! They are not needed. This means you cannot use
\texttt{ListBuffer}s, \texttt{Array}s, for example. Do not use
\texttt{return} in your code! It has a different meaning in Scala,
than in Java.  Do not use \texttt{var}! This declares a mutable
variable.  

\subsection*{Disclaimer}

It should be understood that the work you submit represents your own
effort! You have not copied from anyone or anywhere else. An exception
is the Scala code I showed during the lectures or uploaded to KEATS,
which you can freely use.\bigskip


\subsection*{Background}

\noindent
The fields in the file \texttt{log.csv} are organised 
as follows:

\begin{center}
\texttt{counter, id, time\_date, name, country, parent\_id, msg}
\end{center}  

\noindent
Each line in this file contains the data for a single message.  The field
\texttt{counter} is an integer number given to each message; \texttt{id} is a
unique hash string for a message; \texttt{time\_date} is the time when the message
was sent; \texttt{name} and \texttt{country} is data about the author
of the message, whereby sometimes the authors left the country information
empty; \texttt{parent\_id} is a hash specifying which other message the
message answers (this can also be empty). \texttt{Msg} is the actual
message text. \textbf{Be careful} for the tasks below that this text can contain
commas and needs to be treated special when the line is split up
by using \texttt{line.split(",").toList}. Tasks (2) and (3) are about
processing this data and storing it into the \texttt{Rec}-data-structure, which
is pre-defined in the file \texttt{resit.scala}:


\begin{center}
\begin{verbatim}  
  Rec(num: Int, 
      msg_id: String,
      date: String,
      msg: String,
      author: String,
      country: Option[String],
      reply_id : Option[String],
      parent: Option[Int] = None,
      children: List[Int] = Nil)  
\end{verbatim}
\end{center}

\noindent
The transformation into a Rec-data-structure is a two-step process
where first the fields for parents and children are given default
values. This information is then filled in in a second step.

The main information that will be computed in the tasks below is from
which country authors are and how many authors are from each
country. The last task will also rank which messages have been the most
popular in terms of how many replies they received (this will computed
according to be the number children, grand-children and so on of a
message).

\subsection*{Tasks}


\begin{itemize}
\item[(1)] The function \texttt{get\_csv} takes a file name as
  argument. It should read the corresponding file and return its
  content. The content should be returned as a list of strings, namely a
  string for each line in the file. Since the file is a csv-file, the
  first line (the header) should be dropped in the result. Lines are
  separated by \verb!"\n"!. For the file \texttt{log.csv} there should
  be a list of 680 separate strings.

  \mbox{}\hfill[5\% Marks]

 
\item[(2)] The function \texttt{process\_line} takes a single line
  from the csv-file (as generated by \texttt{get\_csv}) and creates a
  Rec(ord) data structure. This data structure is pre-defined in the
  Scala file.

  For processing a line, you should use the function

  \begin{center}
    \verb!<<some_line>>.split(",").toList!
  \end{center}

  \noindent
  in order to separate the fields. HOWEVER BE CAREFUL that the message
  text in the last field of \texttt{log.cvs} can contain commas and
  therefore the split will not always result in a list of only 7
  elements. You need to concatenate anything beyond the 7th field into
  a single string before assigning the field \texttt{msg}.

  \mbox{}\hfill[10\% Marks]

\item[(3)] Each record in the log contains a unique hash code
  identifying each message. For example

  \begin{center}
  \verb!"5ebeb459ac278d01301f1497"!
  \end{center}  

  \noindent
  Some messages also contain a hash code identifying the parent
  message (that is to which question they reply).  The function
  \texttt{post\_process} fills in the information about potential
  children and a potential parent message.
  
  The auxiliary function \texttt{get\_children} takes a record
  \texttt{e} and a record list \texttt{rs} as arguments, and returns
  the list of all direct children (children have the hash code of
  \texttt{e} as \texttt{reply\_id}). The list of children is returned
  as a list of \texttt{num}s. The \texttt{num}s can be used later
  as indexes in a Rec-list.
      
  The auxiliary function \texttt{get\_parent} returns the number of
  the record corresponding to the \texttt{reply\_id} (encoded as
  \texttt{Some} if there exists one, otherwise it returns \texttt{None}).

  In order to update a record, say \texttt{r}, with some additional
  information, you can use the Scala code
  \begin{verbatim}
      r.copy(parent = ....,
             children = ....)
  \end{verbatim}

  \mbox{}\hfill[10\% Marks]
  
\item[(4)] The functions \texttt{get\_countries} and
  \texttt{get\_countries\_numbers} calculate the countries where
  message authors are coming from and how many authors come from each
  country (returned as a \texttt{Map} from countries to Integers). In
  case an author did not specify a country, the empty string should
  be returned.

  \mbox{}\hfill[10\% Mark]


\item[(5)] This task identifies the most popular questions in the log,
  whereby popularity is measured in terms of how many follow-up
  questions were asked. We call such questions as belonging to a
  \emph{thread}. It can be assumed that in \texttt{log.csv} there are
  no circular references, that is no question refers to a
  follow-up question as parent.

  The function \texttt{ordered\_thread\_sizes} orders the
  message threads according to how many answers were given for one
  message (that is how many children, grand-children and so on one
  message has).

  The auxiliary function \texttt{search} enumerates all children,
  grand-children and so on for a given record \texttt{r} (including
  the record \texttt{r} itself). \texttt{Search} returns these children
  as a list of \texttt{Rec}s.

  The function \texttt{thread\_size} generates for a record, say
  \texttt{r}, a pair consisting of the number of \texttt{r} and the
  number of all children as produced by search. The numbers are the
  integers given for each message---for \texttt{log.cvs} a number
  is between 0 and 679.

  The function \texttt{ordered\_thread\_sizes} orders the list of
  pairs according to which thread in the chat is the longest (the
  longest should be first).

\mbox{}\hfill[15\% Mark]
\end{itemize}

\end{document}

  

\end{document}


%%% Local Variables: 
%%% mode: latex
%%% TeX-master: t
%%% End: