11 part are due on 16 November at 11pm, and the third part on 21 December

    10 part are due on 16 November at 11pm, and the third part on 23 November

    15 \bigskip

    14 Make sure the files you submit can be processed by just calling

    16 

    15 \texttt{scala <<filename.scala>>}.\bigskip

    17 \IMPORTANT{}

    16 

    18 

    17 \noindent

    19 \noindent

    18 \textbf{Important:} Do not use any mutable data structures in your

    20 Also note that the running time of each part will be restricted to a

    19 submissions! They are not needed. This means you cannot use 

    21 maximum of 360 seconds on my laptop.

    20 \texttt{ListBuffer}s, for example. Do not use \texttt{return} in your

    22 

    21 code! It has a different meaning in Scala, than in Java.

    23 \DISCLAIMER{}

    22 Do not use \texttt{var}! This declares a mutable variable. Make sure the

    23 functions you submit are defined on the ``top-level'' of Scala, not

    24 inside a class or object. Also note that the running time of

    25 each part will be restricted to a maximum of 360 seconds on my laptop.

    26 

    27 

    28 \subsection*{Disclaimer}

    29 

    30 It should be understood that the work you submit represents

    31 your own effort. You have not copied from anyone else. An

    32 exception is the Scala code I showed during the lectures or

    33 uploaded to KEATS, which you can freely use.\bigskip

    80   $1$ Million. \hfill[2 Marks]

    91   $1$ million. \hfill[2 Marks]

   100 \item 1 to 1 Million where $837,799$ takes 525 steps

   111 \item 1 to 1 million where $837,799$ takes 525 steps

   103   

   186 

   104 \noindent

   187 \noindent

   105 \textbf{Hints:} useful math operators: \texttt{\%} for modulo; useful

   188 \textbf{Test Data:}

   106 functions: \mbox{\texttt{(1\,to\,10)}} for ranges, \texttt{.toInt},

   189 In case of Google, the financial data records 3077 entries starting

   107 \texttt{.toList} for conversions, \texttt{List(...).max} for the

   190 from 2004-08-19 until 2016-11-04 (which is the last entry on the day

   108 maximum of a list, \texttt{List(...).indexOf(...)} for the first index of

   191 when I prepared the course work...namely on 6 November; remember stock

   109 a value in a list.

   192 markets are typically closed on weekends and no financial data is

   110 

   193 produced then; also I did not count the header line). The lowest

   111 

   194 shareprice for Google was on 2004-09-03 with \$49.95513 per share and the

   112 

   195 highest on 2016-10-24 with \$813.109985 per share.\bigskip

   113 \subsection*{Part 2 (3 Marks)}

   196 

   114 

   197 \subsection*{Advanced Part 3 (3 Marks)}

   115 This part is about web-scraping and list-processing in Scala. It uses

   116 online data about the per-capita alcohol consumption for each country

   117 (per year?), and a file containing the data about the population size of

   118 each country.  From this data you are supposed to estimate how many

   119 litres of pure alcohol are consumed worldwide.\bigskip

   120 

   121 \noindent

   122 \textbf{Tasks (file alcohol.scala):}

   123 

   124 \begin{itemize}

   125 \item[(1)] Write a function that given an URL requests a

   126   comma-separated value (CSV) list.  We are interested in the list

   127   from the following URL

   128 

   129 \begin{center}

   130   \url{https://raw.githubusercontent.com/fivethirtyeight/data/master/alcohol-consumption/drinks.csv}

   131 \end{center}

   132 

   133 \noindent Your function should take a string (the URL) as input, and

   134 produce a list of strings as output, where each string is one line in

   135 the corresponding CSV-list.  This list from the URL above should

   136 contain 194 lines.\medskip

   137 

   138 \noindent

   139 Write another function that can read the file \texttt{population.csv}

   140 from disk (the file is distributed with the coursework). This

   141 function should take a string as argument, the file name, and again

   142 return a list of strings corresponding to each entry in the

   143 CSV-list. For \texttt{population.csv}, this list should contain 216

   144 lines.\hfill[1 Mark]

   145 

   146 

   147 \item[(2)] Unfortunately, the CSV-lists contain a lot of ``junk'' and we

   148   need to extract the data that interests us.  From the header of the

   149   alcohol list, you can see there are 5 columns

   150   

   151   \begin{center}

   152     \begin{tabular}{l}

   153       \texttt{country (name),}\\

   154       \texttt{beer\_servings,}\\

   155       \texttt{spirit\_servings,}\\

   156       \texttt{wine\_servings,}\\

   157       \texttt{total\_litres\_of\_pure\_alcohol}

   158     \end{tabular}  

   159   \end{center}

   160 

   161   \noindent

   162   Write a function that extracts the data from the first column,

   163   the country name, and the data from the fifth column (converted into

   164   a \texttt{Double}). For this go through each line of the CSV-list

   165   (except the first line), use the \texttt{split(",")} function to

   166   divide each line into an array of 5 elements. Keep the data from the

   167   first and fifth element in these arrays.\medskip

   168 

   169   \noindent

   170   Write another function that processes the population size list. This

   171   is already of the form country name and population size.\footnote{Your

   172     friendly lecturer already did the messy processing for you from the

   173   Worldbank database, see \url{https://github.com/datasets/population/tree/master/data} for the original.} Again, split the

   174   strings according to the commas. However, this time generate a

   175   \texttt{Map} from country names to population sizes.\hfill[1 Mark]

   176 

   177 \item[(3)] In (2) you generated the data about the alcohol consumption

   178   per capita for each country, and also the population size for each

   179   country. From this generate next a sorted(!) list of the overall

   180   alcohol consumption for each country. The list should be sorted from

   181   highest alcohol consumption to lowest. The difficulty is that the

   182   data is scraped off from ``random'' sources on the Internet and

   183   annoyingly the spelling of some country names does not always agree in both

   184   lists. For example the alcohol list contains

   185   \texttt{Bosnia-Herzegovina}, while the population writes this country as

   186   \texttt{Bosnia and Herzegovina}. In your sorted

   187   overall list include only countries from the alcohol list, whose

   188   exact country name is also in the population size list. This means

   189   you can ignore countries like Bosnia-Herzegovina from the overall

   190   alcohol consumption. There are 177 countries where the names

   191   agree. The UK is ranked 10th on this list by

   192   consuming 671,976,864 Litres of pure alcohol each year.\medskip

   193   

   194   \noindent

   195   Finally, write another function that takes an integer, say

   196   \texttt{n}, as argument. You can assume this integer is between 0

   197   and 177 (the number of countries in the sorted list above).  The

   198   function should return a triple, where the first component is the

   199   sum of the alcohol consumption in all countries (on the list); the

   200   second component is the sum of the \texttt{n}-highest alcohol

   201   consumers on the list; and the third component is the percentage the

   202   \texttt{n}-highest alcohol consumers drink with respect to the

   203   the world consumption. You will see that according to our data, 164

   204   countries (out of 177) gobble up 100\% of the World alcohol

   205   consumption.\hfill\mbox{[1 Mark]}

   206 \end{itemize}

   207 

   208 \noindent

   209 \textbf{Hints:} useful list functions: \texttt{.drop(n)},

   210 \texttt{.take(n)} for dropping or taking some elements in a list,

   211 \texttt{.getLines} for separating lines in a string;

   212 \texttt{.sortBy(\_.\_2)} sorts a list of pairs according to the second

   213 elements in the pairs---the sorting is done from smallest to highest;

   214 useful \texttt{Map} functions: \texttt{.toMap} converts a list of

   215 pairs into a \texttt{Map}, \texttt{.isDefinedAt(k)} tests whether the

   216 map is defined at that key, that is would produce a result when

   217 called with this key; useful data functions: \texttt{Source.fromURL},

   218 \texttt{Source.fromFile} for obtaining a webpage and reading a file.

   219 

   220 \newpage

   221 

   222 \subsection*{Advanced Part 3 (4 Marks)}

   230 Since the question about Mr Drumb's business acumen remains open,

   205 Since the question about Mr Drumb's business acumen remains, let's do a

   231 let's do a quick back-of-the-envelope calculation in Scala whether his

   206 quick back-of-the-envelope calculation in Scala whether his claim has

   232 claim has any merit. Let's suppose we are given \$100 in 1978 and we

   207 any merit. Let's suppose we are given \$100 in 1978 and we follow a

   233 follow a really dumb investment strategy, namely:

   208 really dumb investment strategy, namely:

   247   some companies went bust or were de-listed).

   222   some companies went bust or de-listed).

   248 \item We do this for 39 years until January 2017 and check what would

   223 \item We do this for 38 years until January 2016 and check what would

   250 \end{itemize}

   225 \end{itemize}\medskip  

   251 

   252 \noindent

   253 Until Yahoo was bought by Altaba this summer, historical stock market

   254 data for such back-of-the-envelope calculations was freely available

   255 online. Unfortuantely nowadays this kind of data is difficult to

   256 obtain, unless you are prepared to pay extortionate prices or be

   257 severely rate-limited.  Therefore this coursework comes with a number

   258 of files containing CSV-lists with the historical stock prices for the

   259 companies in our portfolios. Use these files for the following

   260 tasks.\bigskip

   266 \item[(1.a)] Write a function \texttt{get\_january\_data} that takes a

   231 \item[(1.a)] Write a function that queries the Yahoo financial data

   267   stock symbol and a year as arguments. The function reads the

   232   service and obtains the first trade (adjusted close price) of a

   268   corresponding CSV-file and returns the list of strings that start

   233   stock symbol and a year. A problem is that normally a stock exchange

   269   with the given year (each line in the CSV-list is of the form

   234   is not open on 1st of January, but depending on the day of the week

   270   \texttt{year-01-someday,someprice}).

   235   on a later day (maybe 3rd or 4th). The easiest way to solve this

   271 

   236   problem is to obtain the whole January data for a stock symbol as

   272 \item[(1.b)] Write a function \texttt{get\_first\_price} that takes

   237   CSV-list and then select the earliest entry in this list. For this

   273   again a stock symbol and a year as arguments. It should return the

   238   you can specify a date range with the Yahoo service. For example if

   274   first January price for the stock symbol in given the year. For this

   239   you want to obtain all January data for Google in 2000, you can form

   275   it uses the list of strings generated by

   240   the query:\mbox{}\\[-8mm]

   276   \texttt{get\_january\_data}.  A problem is that normally a stock

   241 

   277   exchange is not open on 1st of January, but depending on the day of

   242   \begin{center}\small

   278   the week on a later day (maybe 3rd or 4th). The easiest way to solve

   243     \mbox{\url{http://ichart.yahoo.com/table.csv?s=GOOG&a=0&b=1&c=2000&d=1&e=1&f=2000}}

   279   this problem is to obtain the whole January data for a stock symbol

   244   \end{center}

   280   and then select the earliest, or first, entry in this list. The

   245 

   281   stock price of this entry should be converted into a double.  Such a

   246   For other companies and years, you need to change the stock symbol

   282   price might not exist, in case the company does not exist in the given

   247   (\texttt{GOOG}) and the year \texttt{2000} (in the \texttt{c} and

   283   year. For example, if you query for Google in January of 1980, then

   248   \texttt{f} argument of the query). Such a request might fail, if the

   284   clearly Google did not exist yet.  Therefore you are asked to

   249   company does not exist during this period. For example, if you query

   285   return a trade price as \texttt{Option[Double]}\ldots\texttt{None}

   250   for Google in January of 1980, then clearly Google did not exists yet.

   286   will be the value for when no price exists.

   251   Therefore you are asked to return a trade price as

   287 

   252   \texttt{Option[Double]}.

   288 \item[(1.c)] Write a function \texttt{get\_prices} that takes a

   253 

   289   portfolio (a list of stock symbols), a years range and gets all the

   254 \item[(1.b)] Write a function that takes a portfolio (a list of stock symbols),

   290   first trading prices for each year in the range. You should organise

   255   a years range and gets all the first trading prices for each year. You should

   291   this as a list of lists of \texttt{Option[Double]}'s. The inner

   256   organise this as a list of lists of \texttt{Option[Double]}'s. The inner lists

   292   lists are for all stock symbols from the portfolio and the outer

   257   are for all stock symbols from the portfolio and the outer list for the years.

   293   list for the years.  For example for Google and Apple in years 2010

   258   For example for Google and Apple in years 2010 (first line), 2011

   294   (first line), 2011 (second line) and 2012 (third line) you obtain:

   259   (second line) and 2012 (third line) you obtain:

   297   List(List(Some(311.349976), Some(27.505054)), 

   262   List(List(Some(313.062468), Some(27.847252)), 

   298        List(Some(300.222351), Some(42.357094)),

   263        List(Some(301.873641), Some(42.884065)),

   299        List(Some(330.555054), Some(52.852215)))

   264        List(Some(332.373186), Some(53.509768)))

   300 \end{verbatim}\hfill[2 Marks]

   265 \end{verbatim}\hfill[1 Mark]

   321   List(List(Some(-0.03573992567129673), Some(0.5399749442411563))

   283   List(List(Some(-0.03573991820699504), Some(0.5399747522663995))

   322         List(Some(0.10103412653643493), Some(0.2477771728154912)))

   284           List(Some(0.10103414428290529), Some(0.24777742035415723)))

   326   Both did OK in 2011. Make sure you handle the cases where a company is

   288   Both did OK in 2011.\hfill\mbox{[1 Mark]}

   327   not listed in a year. In such cases the change factor should be \texttt{None}

   328   (see 2.a).\\

   329   \mbox{}\hfill\mbox{[1 Mark]}

   338   received in our running example involving Google and Apple:

   297   received in our running example

   341   $50 * -0.03573992567129673 + $50 * 0.5399749442411563

   300   $50 * -0.03573991820699504 + $50 * 0.5399747522663995

   342                                        = $25.21175092849298

   301                                          = $25.211741702970222

   350   the new balances and then re-invested into our portfolio.\\

   309   the new balances and then re-invested into our portfolio.\mbox{}\hfill\mbox{[1 Mark]}

   351   \mbox{}\hfill\mbox{[1 Mark]}

   357 Facebook, Amazon and Baidu; and another for listed real-estate

   315 Facebook, Amazon and Baidu; and another for listed real-estate companies, whose

   358 companies, whose names I have never heard of. Following the dumb

   316 names I have never heard of. Following the dumb investment strategy

   359 investment strategy from 1978 until 2017 would have turned a starting

   317 from 1978 until 2016 would have turned a starting balance of \$100

   360 balance of \$100 into roughly \$30,895 for real estate and a whopping

   318 into \$23,794 for real estate and a whopping \$524,609 for blue chips.\medskip

   361 \$349,597 for blue chips.  Note when comparing these results with your

   362 own calculations: there might be some small rounding errors, which

   363 when compounded lead to moderately different values.\bigskip

   364 

   365 \noindent

   366 \textbf{Hints:} useful string functions: \texttt{.startsWith(...)} for

   367 checking whether a string has a given prefix, \texttt{\_ ++ \_} for

   368 concatenating two strings; useful option functions: \texttt{.flatten}

   369 flattens a list of options such that it filters way all

   370 \texttt{None}'s, \texttt{Try(...) getOrElse ...} runs some code that

   371 might raise an exception---if yes, then a default value can be given;

   372 useful list functions: \texttt{.head} for obtaining the first element

   373 in a non-empty list, \texttt{.length} for the length of a

   374 list.\bigskip

   375 

   385 equally well, if not much better.\medskip

   328 equally well, if not much better.

   386 

   329 

   387 

   330 

   331 About rounding errors: \url{https://www.youtube.com/watch?v=pQs_wx8eoQ8}

   332 (PBS Infinity Series).