Big Data Managed Services in the Cloud – Lesson Summary
That concludes the you have data but what you going to do with it module. Let me reminf you of what you have learned so far. Cloud dataproc provides a fast, easy cost effective way to run Apache Hadoop and Apache spark which are open source big data technologies that support big data operations. Cloud dataproc use cases include helping with log processing, adhoc data analysis, and even machine learning. Cloud data flow uses the Apache Beam SDK to offer a simplified stream and batch dataflow processing pipeline. You use cloud dataflow to build those data pipelines, monitor their execution and then transform and analyse that data. Remember the discussion on sources and sinks. Cloud dataflow templates for the rapid deployment of job types. The BigQuery service replaces the typical hardware setup for a traditional data warehouse and again it serves as a collective home for all of your analytical data within your organization.