ETL is a sort of information incorporation measure alluding to three particular yet interrelated advances (Extract, Transform and Load) and is utilized to combine information from various sources commonly to construct a Data Warehouse, Data Hub, or Data Lake.
The most well-known mix-up and misinterpretation made when planning and building an ETL arrangement is bouncing into purchasing new devices and composing code prior to having a complete comprehension of business prerequisites/needs.
There are some things that need to be known before implementing an ETL solution.
For what reason Do We Need ETL?
Start with the business objective.that is
- deep historical context for business.
- Just in time view context for business.
- Improve and learn from the historical data
- Predict the future of the business.
To achieve all of this, we need clean and properly formatted data for data science or business teams to work with. And that is when the ETL process shines.
The basic of ETL process is
1.Data Extraction
2.Data Cleansing
3.Data Transformation
4.Data Loading
To achieve all of this, you need to start with the goal in mind.
1. What is the business objective?
2. Which data segment that we need to achieve first.
3. Which data source do we need to achieve the second.
4. Understand the data source from the third.
5. Choose a suitable cleansing mechanism for each data source.
6. Load the data.
7. Use the data to achieve the first objective.