Massive Data Storage, Retrieval and Deep Learning

Course Description

In today's world of massive amounts of data, new methods and techniques are needed to dive deeper into such data including hands-on practical work on real data sets. This course covers the major existing algorithmic primitives necessary for the processing of "massive" data sets in external memory and data streaming settings. It also introduces various deep learning frameworks and corresponding neural methods in a variety of application areas. Students have the opportunity to gain experience with major platforms for storage and retrieval, including Hadoop Distributed File Systems and Data Streaming with Apache Spark. The focus of one "large data" project is on techniques for information retrieval, natural language processing, and the necessary indexing methods for the retrieval of semantically relevant information. A second "large data" project focuses on the application of a deep learning framework to an application area of each student's choice. This class is related to CS541 (graduate databases) and to CS539 (physical databases) – but it is much systems oriented and focused on practical hands on aspects of storing, processing, and managing massive amounts of structured and unstructured data. Prerequisites: Basic familiarity with data structures and algorithms (like the one provided in CS512), mathematical foundations and probability theory (CS501), and data base foundations (like CS527 and/or CS539). Some of the tools to be used require knowledge of Python, C++, Java, or Scala. Other examples will be based on Apache Spark. However, prior knowledge of Spark (using Scala) is not required but certainly it will not be harmful.

Code

16:198:543

Credits

3

Subject

Computer Science