All the Tech stuff I learn or do
EL, ELT, ETL EL : Extract and load ELT : Extract, Load, Transform ETL : Extract, Transform, Load What are the purposes of Data Quality processing? Validity : Data conforms to your business rules Accuracy : Data conforms to an objective true value Completness : Create, Save and store datasets Read more…
Introduction to Data Engineering Role of a data engineer ->building data pipelines Get the data to where it can be useful Get the data into a usable condition Add new value to the data Manage the data Productionize data processes A data lake brings together data from across the enterprise Read more…
Stream analytics has many applications Data Integration Access data warehousesin real-time Take the load off scource databases with change data capture (CDC) Utilize microservices more effectively Online decisions Real-time recommandations Fraud detection Gaming events Finance back office apps Volume (Terabites of data) + Velocity (Real time) + Variety (Unstructured data) Read more…
Based on the course Introduction to blockchain Technologies by Insead and Don Tapscott / Alex Tapscott on Coursera Principle 1 : Networked integrity Honesty, consideration, accountability, transparency are coded into the blockchain. Integrity is distributed among nodes, not vested in a single member. Double-spend problem. cannot send the same dollar Read more…
We will use the data from the world bank. It contains data on world economies for over half a century. The indicators are : Population Electricity consumption CO2 emissions Literacy rates Unemployment Mortality rates Dictionaries for data science+ Writing a function to help you Using a list comprehension Turning this Read more…
Populate a list with a for loop : We can do that in only one line with a list comprehension ! Now a list comprehension with range() : List comprehensions : Collapse for loops for building lists into a single line Components : iterable iterator variable (represent members of iterable) Read more…
We can iterate over a list using a for loop We can iterate over a string using a for loop We can iterate over a range object using a for loop Iterators vs iterables : Iterable : Examples : lists, strings, dictionaries, file connections An object with an associated iter() Read more…
Module 01 : What is Machine Learning (ML)? Machine learning is a way to use standard algorithms to derive predictive insights from data and make repeated decisions. Business intelligence is understand the past Machine learning use past data to predict the future Stage 1 : Train an Model model with Read more…
There is a quicker way to write functions: lambdas ! So basically you specify the variables in input, and then the operation which is performed et.. voila ! You can also write anonymous functions : Function map takes two arguments: map(func, seq) map() applies the function to ALL elements in Read more…
Crash course on scope in functions Not all objects are accessible everywhere in a script. Scope – part ofthe program where an object or name may be accessible Global scope – defined in the main body of a script Local scope – defined inside a function Built-in scope – names Read more…