Finally after a few month spent studying I decided to jump the fence and register for the exam. Here is how I prepared and what I found the most useful. First I studied the cursus offered on Coursera, the 5 modules. As I never touched a cloud before and had Read more…
Selecting Appropriate Storage Technologies Know the four stages of the data lifecycle: ingest, storage, process and analyze, and explore and visualize. Ingestion is the process of bringing application data, streaming data, and batch data into the cloud. The storage stage focuses on persisting data to an appropriate storage system. Processing Read more…
Practice Exam Q1 : Storage of JSON files with occasionally changing schema, for ANSI SQL queries. This is correct because of the requirement to support occasionally (schema) changing JSON files and aggregate ANSI SQL queries: you need to use BigQuery, and it is quickest to use ‘Automatically detect’ for schema Read more…
Preparing for Data Designing data processing systems System availability is important to pipeline processing, but not data representation, and capacity is important to processing, but not the abstract pipeline or the representation. Think about data engineering and Google Cloud as a platform consisting of components that can be assembled into solutions. Let’s review the elements of GCP that form the Read more…
What is ML ? ML is a way to get predictive insights from data to make repeated decisions off of. You can train the software to estimate the amount of taxes that you owe. Or train that same software to estimate the amount of time it will take to get you home. The ML Read more…
EL, ELT, ETL EL : Extract and load ELT : Extract, Load, Transform ETL : Extract, Transform, Load What are the purposes of Data Quality processing? Validity : Data conforms to your business rules Accuracy : Data conforms to an objective true value Completness : Create, Save and store datasets Read more…
Introduction to Data Engineering Role of a data engineer ->building data pipelines Get the data to where it can be useful Get the data into a usable condition Add new value to the data Manage the data Productionize data processes A data lake brings together data from across the enterprise Read more…
Stream analytics has many applications Data Integration Access data warehousesin real-time Take the load off scource databases with change data capture (CDC) Utilize microservices more effectively Online decisions Real-time recommandations Fraud detection Gaming events Finance back office apps Volume (Terabites of data) + Velocity (Real time) + Variety (Unstructured data) Read more…
Module 01 : What is Machine Learning (ML)? Machine learning is a way to use standard algorithms to derive predictive insights from data and make repeated decisions. Business intelligence is understand the past Machine learning use past data to predict the future Stage 1 : Train an Model model with Read more…