Cloud Computing

Module Tutor: Dell Zhang
Time: Friday evenings 6pm - 9pm (Spring Term)
Room: Birkbeck Main Building, Malet Street (MAL) 421 [BBK-DCS Teaching Map]
Code: BUCI028H6 (for UG), BUCI029H7 (for PG)
Teaching Assistant: Cosmin Stamate

Facebook Group: Cloud Computing (Birkbeck)

Prerequisites: Good knowledge of object-oriented programming in Python would be necessary.
BSc students should have already taken the Software and Programming II (SP2) module.
MSc students who did not have much experience in software development before joining their respective postgraduate programmes should have already taken the Principles of Programming I (POP1) module.

UCL students: Please note that although this Birkbeck module is taught in the spring term of 2018/19, all UCL students who would like to take it as part of their MSc programme must complete and submit the signed intercollegiate study application form (2018/19) to the admin office (Room 263, Birkbeck Main Building), before the deadline 31st October 2018.
The module can accommodate at most 25 UCL students (first-come, first-served) from MSc WSBDA and MSc DS.

Books

	Jothy Rosenberg and Arthur Mateos, *The Cloud at Your Service*, Manning, 2010. Companion Website		Dan Marinescu, *Cloud Computing: Theory and Practice*, Morgan Kaufmann, 2013. Companion Website eBook in the Birkbeck Library
	Jimmy Lin and Chris Dyer, *Data-Intensive Text Processing with MapReduce*, Morgan and Claypool, 2010. Free eBook Companion Website Code and Data		Serge Abiteboul, Ioana Manolescu, Philippe Rigaux, Marie-Christine Rousset, and Pierre Senellart, *Web Data Management* , Cambridge University Press, 2011. Free eBook Companion Website
	Jure Leskovec, Anand Rajaraman and Jeff Ullman, *Mining of Massive Datasets*, 2nd edition, Cambridge University Press, 2014. Free eBook (3rd edition, beta version) Companion Website		Ian Foster and Dennis Gannon, *Cloud Computing for Science and Engineering*, MIT Press, 2017. Free eBook Companion Website
	Chuck Lam, *Hadoop in Action*, Manning, 2010. Companion Website Code and Data		Tom White, *Hadoop: The Definitive Guide*, 4th edition, O'Reilly Media, 2015. Companion Website Code and Data

Resources and Tools

Amazon Web Services (AWS): AWS Educate; Amazon EC2, and Amazon EMR, etc.
Apache: Hadoop and Spark, etc.

Syllabus

Week	Date	Lecture
1	18/01/2019	Welcome [slides] Introduction to Cloud Computing [slides] What is cloud computing The history of cloud computing[CAYS-1.0] [CAYS-1.3] [CAYS-9.1] The principles of cloud computing [CAYS-1.1] [CCSE-4.1] The benefits of cloud computing [CAYS-1.2] The economics of cloud computing [CAYS-3.1] Where does the cloud (not) make sense [CAYS-3.2,3.3] Cloud Computing Technologies and Types [slides] The technological underpinnings of cloud computing [CAYS-2.1.1] [CAYS-2.1.2] [CAYS-2.1.4] [CAYS-2.1.6] Understanding the different classifications of clouds [CAYS-1.4] [CAYS-2.2] [CCSE-4.3] Matching cloud providers to your needs [CAYS-2.3]
2	25/01/2019	Parallel Computing and Distributed Systems [slides] Parallel Computing: speed-up, parallelism, locking [CCTP-2a] Distributed Systems: CAP theorem, logical clocks, snapshot algorithms, consensus protocols, RESTful APIs [WDM-14a] [CCTP-2a] [CAYS-2.1.3]
3	01/02/2019	Big Data [slides] The challenges and opportunities of big data [DITP-1] [BD-1] [CRA-BDWP] Large-scale distributed file systemsL: GFS (HDFS) etc. [MMDS-2.1] [WDM-14b] [HIA-3.1] [HTDG-3] [SOSP03-ghemawat] Large-scale distributed access structures: Dynamo (Voldemort), BigTable (HBase) etc. [WDM-15] [HIA-11.2] [HTDG-13] [OSDI06-chang] NoSQL databases [CAYS-2.1.5]
4	08/02/2019	MapReduce and Hadoop [slides] MapReduce basics [CAYS-6.2.2] [DITP-2] [MMDS-2.2] [WDM-16.1] [HIA-1.5] [HIA-1.6] [HIA-3.2,3.3] [HTDG-2,5] [CACM08-01-patterson] [CACM08-01-dean] [OSDI04-dean] Hadoop basics [CAYS-6.2.3] [WDM-19] [HIA-1.2] [HIA-1.3] [HIA-1.7] [HIA-2,8] [HTDG-1,4]
5	15/02/2019 ------------	UCL Reading Week: No Cloud Computing Lecture for BBK and UCL Students. Please find below the materials to read. Selected Case Studies FlightCaster, GoodData; Eli Lilly, Washington Post, Virgin Atlantic: [CAYS-3.4,3.5,3.6] New York Times, China Mobile, StumbleUpon, and IBM: [HIA-12] Facebook, Nutch, Rackspace, Cascading, TeraByte Sort, Large Networks: [HTDG-16] Last.fm [slides] Getting Started with Google App Engine (GAE)
6	22/02/2019 BBK ITS Labs MAL 414/415 MAL 416/417	Practical Lab Class (NOT required for UG Students) [slides] [slides] Amazon SageMaker Google Colaboratory Microsoft Azure Notebooks [Unix/Linux Cheatsheet] [JupyterLab Cheatsheet] Running Hadoop in the cloud using Amazon EMR [MLIA-15] [Amazon-EMR Tutorial] mrjob
7	01/03/2019	Developing MapReduce Programs [slides] MapReduce algorithm design [DITP-3] [HIA-4,5,6,7] [HTDG-6,7,8]
8	08/03/2019	Information Retrieval in the Cloud [slides] MapReduce for inverted indexing [DITP-4] [OSDI10-peng]
9	15/03/2019	Link Analysis in the Cloud [slides] MapReduce for graph algorithms [DITP-5] [CISE09-cohen] PageRank and its efficient computation [DITP-5.3] [MLG10-lin] [MMDS-5.1] [MMDS-5.2]
10	22/03/2019	Data Management in the Cloud [slides] MapReduce algorithms for processing relational and matrix data [DITP-6] [MMDS-2.3] [XRDS12-ullman]
11	29/03/2019	Data Analytics and Machine Learning in the Cloud [slides] Apache Spark basics [Spark-intro] [MMDS-2.4.2,3] [CACM16-11-zaharia] [NSDI12-zaharia] [SIGMOD15-armbrust] [OSDI14-gonzalez] [SOSP13-zaharia] Apache Spark MLlib [Spark-guide] [JMLR16-meng] Google TensorFlow etc. [MMDS-2.4.4] [CCSE-10] fast.ai Practical Deep Learning for Coders (v3)
--	Friday 10/05/2019 6pm - 9pm 43 Gordon Sq GOR B04	Revision Lecture (for UG Students) [slides] 2012-13 UG Exam Paper 2013-14 UG Exam Paper 2014-15 UG Exam Paper 2015-16 UG Exam Paper 2016-17 UG Exam Paper 2017-18 UG Exam Paper [extra] [extra]
--	Friday 17/05/2019 6pm - 9pm 43 Gordon Sq GOR B04	Revision Lecture (for PG Students) [slides] 2012-13 PG Exam Paper 2013-14 PG Exam Paper 2014-15 PG Exam Paper 2015-16 PG Exam Paper 2016-17 PG Exam Paper 2017-18 PG Exam Paper [extra] [extra]
--	------------	More on Data Management in the Cloud (Optional self-study material, excluded from assessment.) MapReduce vs parallel DBMS [DBC08-dewitt] [CACM10-01-stonebraker] [CACM10-01-dean] Pig [WDM-16.2] [HIA-10] [HTDG-11] [SIGMOD08-olston] Hive [HIA-11.1] [HTDG-12] CouchDB [WDM-20] Dremel [CACM11-06-franklin] [CACM11-06-melnik] [VLDB10-melnik]
--	------------	More on Machine Learning in the Cloud (Optional self-study material, excluded from assessment.) MapReduce for machine learning on multicore [NIPS06-ng] MapReduce for EM algorithms [DITP-7] Large-scale machine learning at Twitter [SIGMOD12-lin] [KDDE12-lin]
--	------------	Security of Cloud Computing (Optional self-study material, excluded from assessment.) Information security refresher [CAYS-a] Security and the private cloud [CAYS-4] Computing arbitrary functions of encrypted data [CACM-10-03-micciancio] [CACM-10-03-gentry]
--	------------	Beyond MapReduce [slides] (Optional self-study material, excluded from assessment.) Extensions to MapReduce [MMDS-2.4] [SIGMOD10-malewicz] [WP-BSP] ~~The communication cost model for MapReduce [MMDS-2.5]~~ ~~The complexity theory for MapReduce [MMDS-2.6]~~ Limitations of MapReduce [DITP-8.1] [AB13-01-31] Alternatives to MapReduce [DITP-8.2] Beyond MapReduce/Hadoop [DITP-8.3] [CACM13-01-Mone]
--	------------	Guest Lectures (2014-2015) Tamas Jambor (Massive Analytic) [slides] [demo] Chris Hillman (Teradata) [slides] Guest Lectures (2013-2014) Richard Conway (Elastacloud) Chris Hillman (Teradata) [slides] Guest Lectures (2012-2013) Roman Kirillov (Google) [slides] [screencast] [code] Mark Levy (Last.fm) [slides]

Assessment

Coursework: 20%

For UG students: One essay of 2,000-3,000 words.
For PG students: Two programming assignments.

Please submit your solutions in electronic form, through the Moodle system.

Normal deadline: Sun 14/04/2019 23:55
Cut-off deadline: Sun 28/04/2019 23:55

Penalty for late submission (i.e., after the normal deadline):
the coursework mark will be capped at the minimum pass mark (i.e., 40% for UG students and 50% for PG students).

Examination: 80%
Past exam papers can be found at Birkbeck eLibrary.

Python Programming

Python [A Short Course for BGRS and BPSN]

Supplements

Research on MapReduce (Literature)