Data Engineering for Beginners: Learn SQL, Python & Spark [2/2025, ENG]

seeders: 15

leechers: 1

Download torrent

Added 4 months ago by unity321d in Other

Main
Technical
Comments

Download Fast Safe Anonymous
movies, software, shows...

Files

Data Engineering for Beginners: Learn SQL, Python & Spark [2/2025, ENG] (Size: 15.33 GB)

	01. Introduction to Data Engineering Essentials using SQL, Python, and PySpark
	1. Introduction to Data Engineering Essentials Course.mp4	4 MB
	1. Introduction to Data Engineering Essentials Course.vtt	3.37 KB
	10. Overview of Performance Tuning of Spark covered in the course.mp4	8.72 MB
	10. Overview of Performance Tuning of Spark covered in the course.vtt	6.49 KB
	2. Overview of our support to Data Engineering Essentials course.mp4	26.3 MB
	2. Overview of our support to Data Engineering Essentials course.vtt	6.02 KB
	3. Overview of SQL topics covered in the course.mp4	36.12 MB
	3. Overview of SQL topics covered in the course.vtt	8.75 KB
	4. Overview of Python topics covered in the course.mp4	37.47 MB
	4. Overview of Python topics covered in the course.vtt	8.7 KB
	5. Overview of Getting Started with GCP related to the course.mp4	17.75 MB
	5. Overview of Getting Started with GCP related to the course.vtt	5.14 KB
	6. Overview of Spark and Databricks Environment related topics.mp4	23.84 MB
	6. Overview of Spark and Databricks Environment related topics.vtt	6.4 KB
	7. Detailed outline of Spark SQL Topics in the course.mp4	46.9 MB
	7. Detailed outline of Spark SQL Topics in the course.vtt	9.18 KB
	8. Detailed outline of Pyspark Topics in the course.mp4	34.78 MB
	8. Detailed outline of Pyspark Topics in the course.vtt	7.05 KB
	9. Detailed outline of ELT Data Pipelines on Databricks.mp4	15.16 MB
	9. Detailed outline of ELT Data Pipelines on Databricks.vtt	3.16 KB
	02. Getting Started with SQL for Data Engineering
	1. Introduction to SQL for Data Engineering.mp4	6.12 MB
	1. Introduction to SQL for Data Engineering.vtt	2.33 KB
	2. Overview of Application Architecture and RDBMS.mp4	16.85 MB
	2. Overview of Application Architecture and RDBMS.vtt	5.57 KB
	3. Overview of Database Technologies and relevance of SQL.mp4	30.27 MB
	3. Overview of Database Technologies and relevance of SQL.vtt	7.98 KB
	4. Overview of Purpose Built Databases.mp4	16.19 MB
	4. Overview of Purpose Built Databases.vtt	8.18 KB
	5. Overview of Data Warehouse and Data Lake.mp4	17.68 MB
	5. Overview of Data Warehouse and Data Lake.vtt	4.71 KB
	6. Usage of RDBMS and Data Warehouse technologies.mp4	15.01 MB
	6. Usage of RDBMS and Data Warehouse technologies.vtt	4.39 KB
	7. Differences and Similarities between RDBMS and Data Warehouse Technologies.mp4	10.82 MB
	7. Differences and Similarities between RDBMS and Data Warehouse Technologies.vtt	4.93 KB
	03. Setup Tools for Data Engineering Essentials
	1. Introduction to Setting up Tools for Data Engineering Essentials.mp4	8 MB
	1. Introduction to Setting up Tools for Data Engineering Essentials.vtt	3.35 KB
	10. Conclusion of Setting up Tools for Data Engineering Essentials.mp4	5.66 MB
	10. Conclusion of Setting up Tools for Data Engineering Essentials.vtt	2.28 KB
	2. Setup VS Code on Windows.mp4	11.82 MB
	2. Setup VS Code on Windows.vtt	4.61 KB
	3. Setup Python 3.9 on Windows.mp4	19.2 MB
	3. Setup Python 3.9 on Windows.vtt	6.95 KB
	4. Configure Environment Variable PATH for Python on Windows.mp4	12.33 MB
	4. Configure Environment Variable PATH for Python on Windows.vtt	5.73 KB
	5. Overview of learning Python using Python CLI.mp4	10.08 MB
	5. Overview of learning Python using Python CLI.vtt	4.4 KB
	6. Integrate VSCode with Python on Windows.mp4	11.66 MB
	6. Integrate VSCode with Python on Windows.vtt	6.25 KB
	7. Install Postgres 14 on Windows 11.mp4	15.18 MB
	7. Install Postgres 14 on Windows 11.vtt	6.58 KB
	8. Getting Started with pgAdmin on Windows.mp4	7.73 MB
	8. Getting Started with pgAdmin on Windows.vtt	3.56 KB
	9. Getting Started with pgAdmin on Mac.mp4	12.96 MB
	9. Getting Started with pgAdmin on Mac.vtt	6.92 KB
	04. Setup Application Tables and Data in Postgres Database
	1. Overview of Postgres Database Server and pgAdmin.mp4	28.81 MB
	1. Overview of Postgres Database Server and pgAdmin.vtt	11.13 KB
	2. Overview of Database Connection Details.mp4	27.29 MB
	2. Overview of Database Connection Details.vtt	9.38 KB
	3. Overview of Connecting to External Databases using pgAdmin.mp4	10.71 MB
	3. Overview of Connecting to External Databases using pgAdmin.vtt	6.25 KB
	4. Complete-PostgreSQL-Setup-Create-Databases-Fix-Errors-Manage-Users-Like-a-Pro.txt	41 B
	4. Create Application Database and User in Postgres Database Server.mp4	8.96 MB
	4. Create Application Database and User in Postgres Database Server.vtt	5.35 KB
	5. Clone Data Sets from Git Repository for Database Scripts.mp4	10.29 MB
	5. Clone Data Sets from Git Repository for Database Scripts.vtt	4.09 KB
	6. Register Server in pgAdmin using Application Database and User.mp4	18.5 MB
	6. Register Server in pgAdmin using Application Database and User.vtt	5.51 KB
	7. Permission-Denied-for-Schema-Public-in-PostgreSQL-How-to-Fix-It.txt	46 B
	7. Setup Application Tables and Data in Postgres Database.mp4	18.22 MB
	7. Setup Application Tables and Data in Postgres Database.vtt	10.31 KB
	8. Overview of pgAdmin to write SQL Queries.mp4	32.67 MB
	8. Overview of pgAdmin to write SQL Queries.vtt	17.64 KB
	05. Writing Basic SQL Queries
	1. Data Model Diagrams - Sales Data Model.jpg?042148	339.75 KB
	1. Review Data Model Diagram.mp4	18.54 MB
	1. Review Data Model Diagram.vtt	7.21 KB
	10. Outer Joins using SQL Queries.mp4	30.03 MB
	10. Outer Joins using SQL Queries.vtt	9.83 KB
	10. Outer-Joins-using-SQL-Queries.txt	200 B
	11. Filter and Aggregate on Join Results using SQL.mp4	10.73 MB
	11. Filter and Aggregate on Join Results using SQL.vtt	5.42 KB
	12. Overview of Database Views.mp4	15.91 MB
	12. Overview of Database Views.vtt	8.47 KB
	13. Overview of Common Table Expressions or CTEs.mp4	10.44 MB
	13. Overview of Common Table Expressions or CTEs.vtt	5.58 KB
	14. Outer Join with Additional Conditions in SQL Queries.mp4	31.19 MB
	14. Outer Join with Additional Conditions in SQL Queries.vtt	10.5 KB
	15. Explanation about Fix of SQL Queries with Filtering on Outer Join Results.mp4	17.08 MB
	15. Explanation about Fix of SQL Queries with Filtering on Outer Join Results.vtt	5.71 KB
	2. Define Problem Statement for SQL Queries.mp4	6.94 MB
	2. Define Problem Statement for SQL Queries.vtt	2.47 KB
	3. Filtering Data using SQL Queries.mp4	22.28 MB
	3. Filtering Data using SQL Queries.vtt	9.12 KB
	3. Filtering-Data-using-SQL-Queries.txt	203 B
	4. Total Aggregations using SQL Queries.mp4	13.44 MB
	4. Total Aggregations using SQL Queries.vtt	7.68 KB
	4. Total-Aggregations-using-SQL-Queries.txt	207 B
	5. Group By Aggregations using SQL Queries.mp4	20.17 MB
	5. Group By Aggregations using SQL Queries.vtt	10.77 KB
	5. Group-By-Aggregations-using-SQL-Queries.txt	212 B
	6. Order of Execution of SQL Queries.mp4	17.45 MB
	6. Order of Execution of SQL Queries.vtt	9.45 KB
	7. Rules and Restrictions to Group and Filter Data in SQL queries.mp4	13.65 MB
	7. Rules and Restrictions to Group and Filter Data in SQL queries.vtt	7.28 KB
	8. Filter Data based on Aggregated Results using Group By and Having.mp4	11.97 MB
	8. Filter Data based on Aggregated Results using Group By and Having.vtt	5.87 KB
	8. Filter-Data-based-on-Aggregated-Results.txt	212 B
	9. Inner Joins using SQL Queries.mp4	18.17 MB
	9. Inner Joins using SQL Queries.vtt	7.44 KB
	9. Inner-Joins-using-SQL-Queries.txt	200 B
	06. Cumulative Aggregations and Ranking in SQL Queries
	1. Introduction to Cumulative Aggregations and Ranking in SQL Queries.mp4	4.22 MB
	1. Introduction to Cumulative Aggregations and Ranking in SQL Queries.vtt	2.12 KB
	10. Filtering based on Global Ranks using Nested Queries and CTEs in SQL.mp4	18.9 MB
	10. Filtering based on Global Ranks using Nested Queries and CTEs in SQL.vtt	7.24 KB
	10. Filtering-Based-on-Global-Ranks-using-Nested-Queries-and-CTEs-in-SQL.txt	233 B
	11. Filtering based on Ranks per Partition using Nested Queries and CTEs in SQL.mp4	14.01 MB
	11. Filtering based on Ranks per Partition using Nested Queries and CTEs in SQL.vtt	7.03 KB
	11. Filtering-based-on-Ranks-with-in-each-key-using-Nested-Queries-and-CTEs-in-SQL.txt	234 B
	12. Create Students table with Data for ranking using SQL.mp4	7.61 MB
	12. Create Students table with Data for ranking using SQL.vtt	3.96 KB
	12. Create-Students-Table-with-Data-for-Ranking-using-SQL.txt	233 B
	13. Difference between rank and dense rank using SQL.mp4	14.76 MB
	13. Difference between rank and dense rank using SQL.vtt	8.41 KB
	13. Difference-between-rank-and-dense-rank-using-SQL.txt	242 B
	2. Overview of CTAS to create tables based on Query Results.mp4	12.2 MB
	2. Overview of CTAS to create tables based on Query Results.vtt	6.36 KB
	3. Create Tables for Cumulative Aggregations and Ranking.mp4	6.04 MB
	3. Create Tables for Cumulative Aggregations and Ranking.vtt	2.84 KB
	3. Create-Tables-for-Cumulative-Aggregations-and-Ranking.txt	259 B
	4. Overview of OVER and PARTITION BY Clause in SQL Queries.mp4	19.7 MB
	4. Overview of OVER and PARTITION BY Clause in SQL Queries.vtt	9.82 KB
	4. Overview-of-OVER-and-PARTITION-BY-Clause.txt	237 B
	5. Compute Total Aggregation using OVER and PARTITION BY in SQL Queries.mp4	5.24 MB
	5. Compute Total Aggregation using OVER and PARTITION BY in SQL Queries.vtt	2.62 KB
	5. Compute-Total-Aggregation-using-OVER-and-PARTITION-BY-in-SQL-Queries.txt	261 B
	6. Overview of Ranking in SQL.mp4	8.87 MB
	6. Overview of Ranking in SQL.vtt	4.6 KB
	6. Overview-of-Ranking-in-SQL.txt	217 B
	7. Compute Global Ranks using SQL.mp4	11.79 MB
	7. Compute Global Ranks using SQL.vtt	5.97 KB
	7. Compute-Global-Ranks-using-SQL.txt	218 B
	8. Compute Ranks based on key using SQL.mp4	12.83 MB
	8. Compute Ranks based on key using SQL.vtt	6.18 KB
	8. Compute-Ranks-based-on-key-using-SQL.txt	228 B
	9. Rules and Restrictions to Filter Data based on Ranks in SQL.mp4	10.27 MB
	9. Rules and Restrictions to Filter Data based on Ranks in SQL.vtt	4.18 KB
	07. SQL Troubleshooting and Debugging Guide
	1. Introduction to SQL Troubleshooting and Debugging Guide.mp4	4.23 MB
	1. Introduction to SQL Troubleshooting and Debugging Guide.vtt	2.61 KB
	10. Troubleshooting Semantec Errors in SQL Queries.mp4	17.1 MB
	10. Troubleshooting Semantec Errors in SQL Queries.vtt	8.04 KB
	11. Overview of Bugs in SQL Queries.mp4	9.38 MB
	11. Overview of Bugs in SQL Queries.vtt	4.59 KB
	12. Development Best Practices with tips to troubleshoot SQL bugs.mp4	13.44 MB
	12. Development Best Practices with tips to troubleshoot SQL bugs.vtt	5.87 KB
	13. Develop Initial Solution based on the requirement.mp4	12.28 MB
	13. Develop Initial Solution based on the requirement.vtt	6.26 KB
	14. Identify and Troubleshoot Bugs in SQL Queries.mp4	56.59 MB
	14. Identify and Troubleshoot Bugs in SQL Queries.vtt	12.03 KB
	15. Develop Solution using Development Best Practices.mp4	33.23 MB
	15. Develop Solution using Development Best Practices.vtt	6.91 KB
	2. Overview of Database Connectivity Issues.mp4	22.65 MB
	2. Overview of Database Connectivity Issues.vtt	10.19 KB
	3. Validate and Setup Telnet on Mac or PC.mp4	22.89 MB
	3. Validate and Setup Telnet on Mac or PC.vtt	6.67 KB
	4. Validate Connectivity to Database Server using telnet.mp4	8.29 MB
	4. Validate Connectivity to Database Server using telnet.vtt	6.36 KB
	5. Troubleshoot Database Connectivity Issue with Correct Host Details.mp4	13.83 MB
	5. Troubleshoot Database Connectivity Issue with Correct Host Details.vtt	9.95 KB
	6. Current Databases and Users in Postgres Database Server.mp4	11.97 MB
	6. Current Databases and Users in Postgres Database Server.vtt	6.81 KB
	7. Troubleshoot Database Credentials and Permissions Issues.mp4	35.88 MB
	7. Troubleshoot Database Credentials and Permissions Issues.vtt	11.73 KB
	8. Overview of Compilation of SQL Queries.mp4	13.74 MB
	8. Overview of Compilation of SQL Queries.vtt	8.05 KB
	9. Troubleshooting Syntax Errors in SQL Queries.mp4	21.47 MB
	9. Troubleshooting Syntax Errors in SQL Queries.vtt	6.07 KB
	08. Performance Tuning of SQL Queries
	1. Introduction to Performance Tuning of SQL Queries.mp4	9.5 MB
	1. Introduction to Performance Tuning of SQL Queries.vtt	5.06 KB
	10. Performance Testing of SQL Queries using Stored Procedure.mp4	17.16 MB
	10. Performance Testing of SQL Queries using Stored Procedure.vtt	5.99 KB
	11. Add Required Indexes to tune performance of SQL Queries.mp4	26.72 MB
	11. Add Required Indexes to tune performance of SQL Queries.vtt	10.09 KB
	12. Guidelines on adding Indexes on Tables for SQL Queries.mp4	11.09 MB
	12. Guidelines on adding Indexes on Tables for SQL Queries.vtt	5.07 KB
	13. Interpreting the explain plan for SQL Queries using Indexes.mp4	20.95 MB
	13. Interpreting the explain plan for SQL Queries using Indexes.vtt	8.06 KB
	14. Conclusion of Performance Tuning of SQL Queries.mp4	10.94 MB
	14. Conclusion of Performance Tuning of SQL Queries.vtt	3.63 KB
	2. Overview of SQL Compilation Process and Explain Plans.mp4	36.21 MB
	2. Overview of SQL Compilation Process and Explain Plans.vtt	6.05 KB
	3. Generate Explain Plans for SQL Queries.mp4	37.09 MB
	3. Generate Explain Plans for SQL Queries.vtt	7.91 KB
	4. Review Tables used for Performance Tuning of SQL Queries.mp4	23.84 MB
	4. Review Tables used for Performance Tuning of SQL Queries.vtt	11.17 KB
	5. Review Data Storage Internals for Tables and Indexes.mp4	14.16 MB
	5. Review Data Storage Internals for Tables and Indexes.vtt	7.66 KB
	6. Review key terms used in Explain Plans for SQL Queries.mp4	13.3 MB
	6. Review key terms used in Explain Plans for SQL Queries.vtt	5.66 KB
	7. Interpret Explain Plans for Basic SQL Queries.mp4	32.6 MB
	7. Interpret Explain Plans for Basic SQL Queries.vtt	7.85 KB
	8. Review the Common Application Scenarios for Performance Tuning.mp4	25.74 MB
	8. Review the Common Application Scenarios for Performance Tuning.vtt	4.25 KB
	9. Write SQL Queries for Customer Orders.mp4	32.19 MB
	9. Write SQL Queries for Customer Orders.vtt	7.59 KB
	09. Exercises for Basic SQL Queries
	1. Simple Exercises for Filtering and Aggregations.mp4	28.37 MB
	1. Simple Exercises for Filtering and Aggregations.vtt	6.51 KB
	1. Simple-Exercises-for-Filtering-and-Aggregations.txt	213 B
	2. Exercises on Joins and Aggregations using SQL.mp4	11.73 MB
	2. Exercises on Joins and Aggregations using SQL.vtt	4.21 KB
	2. Exercises-on-Joins-and-Aggregations-using-SQL.txt	209 B
	10. Solutions for Basic SQL Queries
	1. Solutions for Filtering and Aggregations.mp4	40.11 MB
	1. Solutions for Filtering and Aggregations.vtt	7.04 KB
	1. Solutions-for-Filtering-and-Aggregations.txt	219 B
	2. Solutions for Filtering and Aggregations.mp4	33.13 MB
	2. Solutions for Filtering and Aggregations.vtt	7.82 KB
	3. Validate Data and Review Data Model Diagram.mp4	22.74 MB
	3. Validate Data and Review Data Model Diagram.vtt	5.05 KB
	4. Solution for Exercise 1 to get Customer Order Count.mp4	53.76 MB
	4. Solution for Exercise 1 to get Customer Order Count.vtt	10.76 KB
	5. Solution for Exercise 2 to get Dormant Customers using Outer Join.mp4	75.54 MB
	5. Solution for Exercise 2 to get Dormant Customers using Outer Join.vtt	15.62 KB
	6. Solution for Exercise 3 to get Revenue Per Customer using Outer Join.mp4	62.29 MB
	6. Solution for Exercise 3 to get Revenue Per Customer using Outer Join.vtt	11.05 KB
	7. Solution for Exercise 4 to get Revenue Per Category.mp4	55.45 MB
	7. Solution for Exercise 4 to get Revenue Per Category.vtt	10.27 KB
	8. Solution for Exercise 5 to get Product Count Per Department.mp4	46.72 MB
	8. Solution for Exercise 5 to get Product Count Per Department.vtt	10.59 KB
	11. Getting Started with Python
	1. Setup Visual Studio Workspace for Python Application Development.mp4	15.04 MB
	1. Setup Visual Studio Workspace for Python Application Development.vtt	9.03 KB
	10. Pre-Defined String Manipulation Functions.mp4	28.73 MB
	10. Pre-Defined String Manipulation Functions.vtt	9.45 KB
	10. Pre-Defined-String-Manipulation-Functions.txt	217 B
	11. Overview of Python Lists.mp4	16.11 MB
	11. Overview of Python Lists.vtt	9.85 KB
	11. Overview-of-Python-Lists.txt	214 B
	12. Loops and Conditions in Python.mp4	20.5 MB
	12. Loops and Conditions in Python.vtt	12.19 KB
	12. Loops-and-Conditions-in-Python.txt	224 B
	13. User Defined Functions in Python.mp4	14.62 MB
	13. User Defined Functions in Python.vtt	9.29 KB
	13. User-defined-functions-in-Python.txt	224 B
	2. Setup Notebook Environment in VS Code Workspace.mp4	23.28 MB
	2. Setup Notebook Environment in VS Code Workspace.vtt	9.41 KB
	3. Overview of VS Code Notebook Environment.mp4	9.07 MB
	3. Overview of VS Code Notebook Environment.vtt	7.01 KB
	4. Overview of Cells in VS Code Notebook.mp4	12.98 MB
	4. Overview of Cells in VS Code Notebook.vtt	9.35 KB
	5. Defining Functions in VS Code Notebooks.mp4	13.45 MB
	5. Defining Functions in VS Code Notebooks.vtt	8.79 KB
	6. Run the Code in VS Code Notebook Cell by Line.mp4	12 MB
	6. Run the Code in VS Code Notebook Cell by Line.vtt	8.28 KB
	7. Constants and Variables in Python.mp4	12.17 MB
	7. Constants and Variables in Python.vtt	8.32 KB
	7. Constants-and-Variables-in-Python.txt	225 B
	8. Overview of Python Data Types.mp4	9.74 MB
	8. Overview of Python Data Types.vtt	5.75 KB
	8. Overview-of-Python-Data-Types.txt	221 B
	9. Getting help on Python Variables and Functions.mp4	32.91 MB
	9. Getting help on Python Variables and Functions.vtt	9.01 KB
	9. Getting-help-on-Python-Variables-and-Functions.txt	242 B
	12. Python Collections for Data Engineering
	1. Overview of File IO using Python.mp4	34.51 MB
	1. Overview of File IO using Python.vtt	11.06 KB
	1. Overview-of-File-IO-using-Python.txt	224 B
	10. Overview of JSON Strings and Files.mp4	13.44 MB
	10. Overview of JSON Strings and Files.vtt	5.63 KB
	10. Overview-of-JSON-Strings-and-Files.txt	226 B
	11. Read JSON Strings to Python dicts or lists.mp4	18.25 MB
	11. Read JSON Strings to Python dicts or lists.vtt	9.37 KB
	12. Read JSON Schemas from file to Python dicts.mp4	26.09 MB
	12. Read JSON Schemas from file to Python dicts.vtt	9.01 KB
	13. Overview of Processing JSON Data using Python.mp4	8.02 MB
	13. Overview of Processing JSON Data using Python.vtt	4.65 KB
	13. Overview-of-Processing-JSON-Data-using-Python.txt	223 B
	14. Extract Details from Complex JSON Arrays using Python.mp4	14.71 MB
	14. Extract Details from Complex JSON Arrays using Python.vtt	5.4 KB
	15. Sort Data in JSON Arrays using Python.mp4	15.41 MB
	15. Sort Data in JSON Arrays using Python.vtt	6.46 KB
	16. Create Function to get Column Details from Schemas JSON File.mp4	23.44 MB
	16. Create Function to get Column Details from Schemas JSON File.vtt	7.9 KB
	2. Read Data from CSV File into Python List.mp4	10.05 MB
	2. Read Data from CSV File into Python List.vtt	5.31 KB
	2. Read-Data-from-CSV-File-into-Python-List.txt	236 B
	3. Overview of Python Collections.mp4	32.79 MB
	3. Overview of Python Collections.vtt	11.99 KB
	3. Overview-of-Python-Collections.txt	218 B
	4. Getting Started with Processing Python Lists.mp4	11.95 MB
	4. Getting Started with Processing Python Lists.vtt	4.46 KB
	4. Getting-Started-with-Processing-Python-Lists.txt	209 B
	5. Overview of Lambda Functions in Python.mp4	15.19 MB
	5. Overview of Lambda Functions in Python.vtt	10 KB
	6. Usage of Lambda Functions.mp4	10.95 MB
	6. Usage of Lambda Functions.vtt	5.29 KB
	7. Filter Data in Python Lists using filter and lambda.mp4	20.15 MB
	7. Filter Data in Python Lists using filter and lambda.vtt	8.36 KB
	8. Get unique values from list using map and set.mp4	21.4 MB
	8. Get unique values from list using map and set.vtt	8.34 KB
	9. Sort Python lists using key.mp4	48 MB
	9. Sort Python lists using key.vtt	10.91 KB
	13. Data Processing using Pandas Dataframe APIs
	1. Overview of Pandas for Data Processing.mp4	28.01 MB
	1. Overview of Pandas for Data Processing.vtt	6.76 KB
	1. Overview-of-Pandas-for-Data-Processing.txt	217 B
	10. Sort Data in Pandas Dataframes.mp4	27.51 MB
	10. Sort Data in Pandas Dataframes.vtt	9.63 KB
	10. Sort-Data-in-Pandas-Dataframes.txt	207 B
	11. Overview of Writing Pandas Dataframes to Files.mp4	9.38 MB
	11. Overview of Writing Pandas Dataframes to Files.vtt	4.71 KB
	12. Write Pandas Dataframes to JSON Files.mp4	65.28 MB
	12. Write Pandas Dataframes to JSON Files.vtt	10.44 KB
	12. Write-Pandas-Dataframes-to-JSON-Files.txt	216 B
	2. Overview of Reading CSV Data using Pandas.mp4	18.03 MB
	2. Overview of Reading CSV Data using Pandas.vtt	6.66 KB
	3. Read Data from CSV Files to Pandas Dataframes.mp4	17.07 MB
	3. Read Data from CSV Files to Pandas Dataframes.vtt	6.61 KB
	3. Read-Data-from-CSV-Files-to-Pandas-Dataframes.txt	227 B
	4. Filter Data in Pandas Dataframe using query.mp4	46.41 MB
	4. Filter Data in Pandas Dataframe using query.vtt	10.37 KB
	4. Filter-Data-in-Pandas-Dataframe-using-query.txt	208 B
	5. Get Count by Status using Pandas Dataframe APIs.mp4	28.93 MB
	5. Get Count by Status using Pandas Dataframe APIs.vtt	7.83 KB
	6. Get count by Month and Status using Pandas Dataframe APIs.mp4	17.53 MB
	6. Get count by Month and Status using Pandas Dataframe APIs.vtt	6.57 KB
	7. Create Dataframes using dynamic column list on CSV Data.mp4	19.99 MB
	7. Create Dataframes using dynamic column list on CSV Data.vtt	7.15 KB
	8. Performing Inner Join between Pandas Dataframes.mp4	28.72 MB
	8. Performing Inner Join between Pandas Dataframes.vtt	6.33 KB
	9. Perform Aggregations on Join results.mp4	12.55 MB
	9. Perform Aggregations on Join results.vtt	7.13 KB
	14. Project 1 - File Format Converter using Python
	1. Project 1 - File Format Converter Handout.html	2.36 KB
	10. Wrapper to Process all Data Sets.mp4	25.19 MB
	10. Wrapper to Process all Data Sets.vtt	9.48 KB
	10. Wrapper-to-Process-all-Data-Sets.txt	220 B
	11. Setup Project for File Format Converter using Python.mp4	10.95 MB
	11. Setup Project for File Format Converter using Python.vtt	5.9 KB
	12. Install Dependencies for the Python Project using pip.mp4	11.54 MB
	12. Install Dependencies for the Python Project using pip.vtt	4.81 KB
	13. Add Core Logic to Python Application.mp4	29.75 MB
	13. Add Core Logic to Python Application.vtt	5.79 KB
	14. Overview of Run-time Arguments and Environment Variables.mp4	5.53 MB
	14. Overview of Run-time Arguments and Environment Variables.vtt	2.8 KB
	15. Using Run Time Arguments in Python Applications.mp4	14.61 MB
	15. Using Run Time Arguments in Python Applications.vtt	7.59 KB
	16. Overview of Environment Variables.mp4	9.98 MB
	16. Overview of Environment Variables.vtt	5.86 KB
	17. Setting Environment Variables on Windows or Mac or Linux.mp4	21.06 MB
	17. Setting Environment Variables on Windows or Mac or Linux.vtt	6.88 KB
	18. Use Environment Variables in Python Applications.mp4	31.11 MB
	18. Use Environment Variables in Python Applications.vtt	7.59 KB
	19. Use Environment Variables in File Format Converter.mp4	36.87 MB
	19. Use Environment Variables in File Format Converter.vtt	13.27 KB
	2. Get File Names to be processed using glob.mp4	24.55 MB
	2. Get File Names to be processed using glob.vtt	8.82 KB
	2. Get-File-Names-to-be-processed-using-glob.txt	218 B
	20. Pass JSON Array as argument to Python Applications.mp4	32.4 MB
	20. Pass JSON Array as argument to Python Applications.vtt	7.92 KB
	21. Pass Data Sets as Run Time Arguments to File Format Converter.mp4	36.29 MB
	21. Pass Data Sets as Run Time Arguments to File Format Converter.vtt	13.16 KB
	22. Exception Handling in Python Applications.mp4	11.22 MB
	22. Exception Handling in Python Applications.vtt	4.89 KB
	23. Raising Exceptions in Python Applications.mp4	24.79 MB
	23. Raising Exceptions in Python Applications.vtt	5.79 KB
	24. Exception Handling in File Format Converter Application.mp4	34.37 MB
	24. Exception Handling in File Format Converter Application.vtt	6.53 KB
	3. Get Column Names using Schemas File.mp4	14.15 MB
	3. Get Column Names using Schemas File.vtt	6.13 KB
	3. Get-Column-Names-using-Schemas-File.txt	223 B
	4. Get Data Set Names from File Names or Paths using regular expressions.mp4	14.61 MB
	4. Get Data Set Names from File Names or Paths using regular expressions.vtt	8.11 KB
	5. Read CSV Data into Pandas Dataframe with Schema Dynamically.mp4	18.13 MB
	5. Read CSV Data into Pandas Dataframe with Schema Dynamically.vtt	7.12 KB
	5. Read-CSV-Data-into-Pandas-Dataframe-with-Schema-Dynamically.txt	239 B
	6. Generate File Paths for Target JSON Files Dynamically.mp4	11.28 MB
	6. Generate File Paths for Target JSON Files Dynamically.vtt	5.49 KB
	6. Generate-File-Paths-for-Target-JSON-Files-Dynamically.txt	222 B
	7. Recap of Writing Pandas Dataframe to JSON File.mp4	9.7 MB
	7. Recap of Writing Pandas Dataframe to JSON File.vtt	3.58 KB
	7. Recap-of-Writing-Pandas-Dataframe-to-JSON-File.txt	238 B
	8. Write Pandas Dataframe to JSON Files.mp4	30.05 MB
	8. Write Pandas Dataframe to JSON Files.vtt	5.98 KB
	8. Write-Pandas-Dataframe-to-JSON-Files.txt	224 B
	9. Modularize File Format Converter for Dataset.mp4	15.42 MB
	9. Modularize File Format Converter for Dataset.vtt	7.91 KB
	9. Modularize-File-Format-Converter-for-Dataset.txt	232 B
	15. Project 2 - Files to Database Loader
	1. 2023-08-07_11-48-28-01319121922bcdac505e6aa6f8d94301.png	14.93 KB
	1. 2023-08-07_11-48-28-ca75f2e65c96a504877e44c5ca19440c.png	530.57 KB
	1. Project 2 - Files To Database Loader Handout.html	2.4 KB
	2. Install Python Dependencies for Pandas and Database Integration.mp4	16.24 MB
	2. Install Python Dependencies for Pandas and Database Integration.vtt	3.25 KB
	2. Install-Python-Dependencies-for-Pandas-and-Database-Integration.txt	219 B
	3. Run Queries from Notebook using SQL Magic.mp4	20.06 MB
	3. Run Queries from Notebook using SQL Magic.vtt	11.59 KB
	3. Run-Queries-from-Notebook-using-SQL-Magic.txt	239 B
	4. Validate Pandas and SQL Integration.mp4	31.72 MB
	4. Validate Pandas and SQL Integration.vtt	7.83 KB
	4. Validate-Pandas-and-SQL-Integration.txt	229 B
	5. Write CSV Data from File to Database Table.mp4	37.16 MB
	5. Write CSV Data from File to Database Table.vtt	8.54 KB
	5. Write-CSV-Data-from-File-to-Database-Table.txt	242 B
	6. Write CSV Data from Files to Database Tables in Chunks.mp4	45.74 MB
	6. Write CSV Data from Files to Database Tables in Chunks.vtt	10.5 KB
	6. Write-CSV-Data-from-Files-to-Database-Tables-in-Chunks.txt	258 B
	7. Overview of Deploying File to DB Loader Project.mp4	37.77 MB
	7. Overview of Deploying File to DB Loader Project.vtt	13.82 KB
	16. Troubleshooting and Debugging Python Issues
	1. Introduction to Troubleshooting and Debugging Python issues.mp4	7.5 MB
	1. Introduction to Troubleshooting and Debugging Python issues.vtt	4.23 KB
	10. Troubleshooting Run Time Errors in Python.mp4	11.5 MB
	10. Troubleshooting Run Time Errors in Python.vtt	5.59 KB
	11. Overview of Software Development Life Cycle.mp4	11.06 MB
	11. Overview of Software Development Life Cycle.vtt	5.21 KB
	12. Overview of Unit Testing or Validation of Applications.mp4	13.15 MB
	12. Overview of Unit Testing or Validation of Applications.vtt	4.85 KB
	13. Overview of Debugging VS Code Notebooks using Debug Feature.mp4	32.67 MB
	13. Overview of Debugging VS Code Notebooks using Debug Feature.vtt	14.04 KB
	14. Debug VS Code Notebooks using Debug Feature.mp4	73.75 MB
	14. Debug VS Code Notebooks using Debug Feature.vtt	14.83 KB
	15. Getting Started with Debugging of Python Programs using VS Code.mp4	9.26 MB
	15. Getting Started with Debugging of Python Programs using VS Code.vtt	3.67 KB
	16. Recap of running File Format Converter application.mp4	26.62 MB
	16. Recap of running File Format Converter application.vtt	9.48 KB
	17. Debug Python Application using VS Code with breakpoints.mp4	33.37 MB
	17. Debug Python Application using VS Code with breakpoints.vtt	9.97 KB
	18. Managing Breakpoints for Debugging in VS Code.mp4	12.41 MB
	18. Managing Breakpoints for Debugging in VS Code.vtt	4.48 KB
	19. Conclusion to Troubleshooting and Debugging Python Issues.mp4	6.44 MB
	19. Conclusion to Troubleshooting and Debugging Python Issues.vtt	3.21 KB
	2. Guidelines for Troubleshooting and Debugging Python related Issues.mp4	5.34 MB
	2. Guidelines for Troubleshooting and Debugging Python related Issues.vtt	2.88 KB
	3. Overview of Database Connectivity using Python Applications.mp4	20.91 MB
	3. Overview of Database Connectivity using Python Applications.vtt	10.26 KB
	4. Overview of Database Connectivity using Python.mp4	11.34 MB
	4. Overview of Database Connectivity using Python.vtt	6.27 KB
	5. Troubleshoot Network Connectivity to the Database Server using telnet.mp4	12.94 MB
	5. Troubleshoot Network Connectivity to the Database Server using telnet.vtt	5.49 KB
	6. Troubleshoot Module Related issues for Database Connectivity using Python.mp4	39.74 MB
	6. Troubleshoot Module Related issues for Database Connectivity using Python.vtt	7.63 KB
	7. Troubleshoot Credentials Related issues for Database Connectivity using Python.mp4	20.82 MB
	7. Troubleshoot Credentials Related issues for Database Connectivity using Python.vtt	4.81 KB
	8. Overview of Python process to run Python Applications.mp4	7.96 MB
	8. Overview of Python process to run Python Applications.vtt	3.86 KB
	9. Troubleshooting Compilation Errors in Python.mp4	13.31 MB
	9. Troubleshooting Compilation Errors in Python.vtt	7.27 KB
	17. Performance Tuning of Python Applications
	1. Introduction to Performance of Python Applications.mp4	20.14 MB
	1. Introduction to Performance of Python Applications.vtt	9.4 KB
	10. Overview of multi or batch insert into Database Tables.mp4	13.4 MB
	10. Overview of multi or batch insert into Database Tables.vtt	7.07 KB
	11. Develop application for multiprocessing.mp4	7.85 MB
	11. Develop application for multiprocessing.vtt	4.31 KB
	12. Getting Started with Multiprocessing using Python.mp4	13.95 MB
	12. Getting Started with Multiprocessing using Python.vtt	7.58 KB
	13. Invoking User Defined Functions using multiprocessing in Python.mp4	25.89 MB
	13. Invoking User Defined Functions using multiprocessing in Python.vtt	8.38 KB
	14. Refactor File to Database Loader Application for Multiprocessing.mp4	31.54 MB
	14. Refactor File to Database Loader Application for Multiprocessing.vtt	6.65 KB
	15. Add Parallel Processing to file to db loader Python Application.mp4	34.4 MB
	15. Add Parallel Processing to file to db loader Python Application.vtt	6.59 KB
	16. Validate File to DB Loader Application with Multiprocessing.mp4	37.95 MB
	16. Validate File to DB Loader Application with Multiprocessing.vtt	9.86 KB
	17. Understanding the concept of Multiprocessing in Python.mp4	27.4 MB
	17. Understanding the concept of Multiprocessing in Python.vtt	10.58 KB
	18. Performance Tuning Scenarios of Python Applications.mp4	15.11 MB
	18. Performance Tuning Scenarios of Python Applications.vtt	6.94 KB
	2. Setup Database Loader Python Application.mp4	22.51 MB
	2. Setup Database Loader Python Application.vtt	8.31 KB
	2. data-engineering-essentials.zip	29.43 MB
	3. Ensure Postgres Database is setup for file to db loader Python Application.mp4	50.33 MB
	3. Ensure Postgres Database is setup for file to db loader Python Application.vtt	13.41 KB
	4. Cleanup the tables to run file to db loader application.mp4	11.28 MB
	4. Cleanup the tables to run file to db loader application.vtt	3.95 KB
	5. Run and Validate File to DB Loader Application.mp4	26.71 MB
	5. Run and Validate File to DB Loader Application.vtt	8.27 KB
	6. Fix the error message in file to db loader application.mp4	18.82 MB
	6. Fix the error message in file to db loader application.vtt	3.46 KB
	7. Overview of Execution of file to db loader application.mp4	21.64 MB
	7. Overview of Execution of file to db loader application.vtt	10.55 KB
	8. Performance Tuning using Chunksize in Pandas.mp4	35.12 MB
	8. Performance Tuning using Chunksize in Pandas.vtt	9.68 KB
	9. Review Pandas Data Frame API to load data into the target table.mp4	54.89 MB
	9. Review Pandas Data Frame API to load data into the target table.vtt	12.55 KB
	18. Getting Started with GCP
	1. Introduction to Getting Started with GCP.mp4	4.18 MB
	1. Introduction to Getting Started with GCP.vtt	1.52 KB
	10. Overview of Google Cloud Shell.mp4	10.29 MB
	10. Overview of Google Cloud Shell.vtt	4.98 KB
	11. Install Google Cloud SDK on Windows.mp4	35.71 MB
	11. Install Google Cloud SDK on Windows.vtt	6.34 KB
	12. Initialize gcloud CLI using GCP Project.mp4	25.65 MB
	12. Initialize gcloud CLI using GCP Project.vtt	4.37 KB
	13. Reinitialize Google Cloud Shell with Project id.mp4	11.69 MB
	13. Reinitialize Google Cloud Shell with Project id.vtt	3.94 KB
	14. Overview of Analytics Services on GCP.mp4	8.26 MB
	14. Overview of Analytics Services on GCP.vtt	3.31 KB
	15. Conclusion to Get Started with GCP for Data Engineering.mp4	8.24 MB
	15. Conclusion to Get Started with GCP for Data Engineering.vtt	1.49 KB
	2. Pre-requisite Skills to Sign up for course on GCP Data Analytics.mp4	5.3 MB
	2. Pre-requisite Skills to Sign up for course on GCP Data Analytics.vtt	3.15 KB
	3. Overview of Cloud Platforms.mp4	8.36 MB
	3. Overview of Cloud Platforms.vtt	6 KB
	4. Overview of Google Cloud Platform or GCP.mp4	13.4 MB
	4. Overview of Google Cloud Platform or GCP.vtt	4.85 KB
	5. Overview of Signing for GCP Account.mp4	5.86 MB
	5. Overview of Signing for GCP Account.vtt	2.57 KB
	6. Create New Google Account using Non Gmail Id.mp4	5.9 MB
	6. Create New Google Account using Non Gmail Id.vtt	3.17 KB
	7. Sign up for GCP using Google Account.mp4	7.74 MB
	7. Sign up for GCP using Google Account.vtt	4.62 KB
	8. Overview of GCP Credits.mp4	21.01 MB
	8. Overview of GCP Credits.vtt	5.1 KB
	9. Overview of GCP Project and Billing.mp4	6.33 MB
	9. Overview of GCP Project and Billing.vtt	3.12 KB
	19. Overview of Big Data and Data Lakes
	1. Different Types of Databases.mp4	4.15 MB
	1. Different Types of Databases.vtt	2.94 KB
	1. Overview of Big Data and Data lakes.pdf	73.77 KB
	10. Implementation of Modern Data Lakes on Cloud.mp4	8.65 MB
	10. Implementation of Modern Data Lakes on Cloud.vtt	5.16 KB
	11. Advantages of Modern Data Lakes on Cloud.mp4	10.47 MB
	11. Advantages of Modern Data Lakes on Cloud.vtt	5.83 KB
	2. Usecases for Different Types of Databases.mp4	9.04 MB
	2. Usecases for Different Types of Databases.vtt	5.74 KB
	3. Technologies for Different Types of Databases.mp4	9.2 MB
	3. Technologies for Different Types of Databases.vtt	5.07 KB
	4. Volumes for Different Types of Databases.mp4	9.16 MB
	4. Volumes for Different Types of Databases.vtt	4.94 KB
	5. Overview of Big Data.mp4	13.81 MB
	5. Overview of Big Data.vtt	6.28 KB
	6. Evolution of Big Data Technologies.mp4	6.05 MB
	6. Evolution of Big Data Technologies.vtt	3.76 KB
	7. Overview of Data Lake using Hadoop eco system.mp4	14.15 MB
	7. Overview of Data Lake using Hadoop eco system.vtt	8.84 KB
	8. Limitations of Hadoop eco system.mp4	12.76 MB
	8. Limitations of Hadoop eco system.vtt	8.13 KB
	9. Overview of Modern Data Lakes on Cloud.mp4	6.54 MB
	9. Overview of Modern Data Lakes on Cloud.vtt	3.77 KB
	20. Overview of Spark and Spark Architecture
	1. Overview of Data Processing.mp4	12.47 MB
	1. Overview of Data Processing.vtt	7.59 KB
	10. Overview of Spark Cluster using Databricks.mp4	29.73 MB
	10. Overview of Spark Cluster using Databricks.vtt	8.44 KB
	11. Overview of Executors in Spark Cluster.mp4	8.34 MB
	11. Overview of Executors in Spark Cluster.vtt	3.76 KB
	12. Overview of Spark Glossary.mp4	10.46 MB
	12. Overview of Spark Glossary.vtt	2.55 KB
	13. Understand Spark Key Terms.mp4	38.75 MB
	13. Understand Spark Key Terms.vtt	10.3 KB
	2. Overview of Data Processing Libraries.mp4	4.22 MB
	2. Overview of Data Processing Libraries.vtt	2.81 KB
	3. Setup Environment to explore Pandas, Dask and Pyspark.mp4	36.85 MB
	3. Setup Environment to explore Pandas, Dask and Pyspark.vtt	6.92 KB
	4. Code Examples of Pandas, Dask and Pyspark.mp4	41.08 MB
	4. Code Examples of Pandas, Dask and Pyspark.vtt	9.75 KB
	5. Differences between Pandas, Dask and Pyspark.mp4	21.09 MB
	5. Differences between Pandas, Dask and Pyspark.vtt	9.43 KB
	6. Overview of Distributed Computing.mp4	13.94 MB
	6. Overview of Distributed Computing.vtt	7.23 KB
	7. Overview of Official Documentation of Apache Spark.mp4	17.22 MB
	7. Overview of Official Documentation of Apache Spark.vtt	5.69 KB
	8. Overview of Spark Key Features and Platforms.mp4	33.88 MB
	8. Overview of Spark Key Features and Platforms.vtt	7.46 KB
	9. Overview of Spark Infrastructure.mp4	19.62 MB
	9. Overview of Spark Infrastructure.vtt	5.29 KB
	21. Setup Databricks Environment using GCP
	1. Overview of Databicks on GCP.mp4	3.4 MB
	1. Overview of Databicks on GCP.vtt	1.75 KB
	10. Configure Databricks CLI on Mac or Windows.mp4	29.7 MB
	10. Configure Databricks CLI on Mac or Windows.vtt	8.21 KB
	11. Troubleshoot issues to configure Databricks CLI.mp4	16.24 MB
	11. Troubleshoot issues to configure Databricks CLI.vtt	7.63 KB
	12. Overview of Databricks CLI Commands.mp4	39.72 MB
	12. Overview of Databricks CLI Commands.vtt	8.51 KB
	13. Setup Data Repository for Data Sets.mp4	15.64 MB
	13. Setup Data Repository for Data Sets.vtt	3.08 KB
	14. Setup Data Sets in DBFS using Databricks CLI Commands.mp4	35.3 MB
	14. Setup Data Sets in DBFS using Databricks CLI Commands.vtt	7.27 KB
	2. Signing up for Databricks on GCP.mp4	16.11 MB
	2. Signing up for Databricks on GCP.vtt	6.18 KB
	3. Create Databricks Workspace on GCP.mp4	9.75 MB
	3. Create Databricks Workspace on GCP.vtt	6.96 KB
	4. Getting Started with Databricks Clusters on GCP.mp4	7.46 MB
	4. Getting Started with Databricks Clusters on GCP.vtt	4.68 KB
	5. Getting Started with Databricks Notebook.mp4	7.34 MB
	5. Getting Started with Databricks Notebook.vtt	5.31 KB
	6. Overview of Databricks on GCP.mp4	24.06 MB
	6. Overview of Databricks on GCP.vtt	10.28 KB
	7. High level architecture of Databricks.mp4	14.72 MB
	7. High level architecture of Databricks.vtt	7.75 KB
	8. Setup Databricks CLI on Mac or Windows.mp4	32.45 MB
	8. Setup Databricks CLI on Mac or Windows.vtt	8.28 KB
	9. Overview of Databricks CLI and other clients.mp4	16.38 MB
	9. Overview of Databricks CLI and other clients.vtt	7.08 KB
	22. Basic Transformations using Spark SQL
	1. Process Data in DBFS using Databricks Spark SQL.mp4	15.67 MB
	1. Process Data in DBFS using Databricks Spark SQL.vtt	6.82 KB
	1. Process-Data-in-DBFS-using-Databricks-Spark-SQL.txt	238 B
	10. Transform Data using Spark APIs.mp4	22.42 MB
	10. Transform Data using Spark APIs.vtt	6.05 KB
	10. Transform-Data-using-Spark-APIs.txt	216 B
	11. Get Schema Details for all Data Sets using Pyspark.mp4	13.05 MB
	11. Get Schema Details for all Data Sets using Pyspark.vtt	5.61 KB
	11. Get-Schema-Details-for-all-Data-Sets-using-Pyspark.txt	243 B
	12. Convert CSV to Parquet with Schema using Pyspark.mp4	29.68 MB
	12. Convert CSV to Parquet with Schema using Pyspark.vtt	6.81 KB
	12. Convert-CSV-to-Parquet-with-Schema-using-Pyspark.txt	239 B
	2. Getting Started with Spark SQL Example using Databricks.mp4	31.74 MB
	2. Getting Started with Spark SQL Example using Databricks.vtt	7.05 KB
	2. Getting-Started-with-Spark-SQL-Example-using-Databricks.txt	246 B
	3. Create Temporary Views using Spark SQL.mp4	35.94 MB
	3. Create Temporary Views using Spark SQL.vtt	9.89 KB
	3. Create-Temporary-Views-using-Spark-SQL.txt	225 B
	4. Exercise to create temporary views using Spark SQL.mp4	5.08 MB
	4. Exercise to create temporary views using Spark SQL.vtt	2.14 KB
	4. Exercise-to-create-temporary-views-using-Spark-SQL.txt	241 B
	5. Spark SQL Query to compute Daily Product Revenue.mp4	22.8 MB
	5. Spark SQL Query to compute Daily Product Revenue.vtt	8.42 KB
	5. Spark-SQL-Query-to-compute-Daily-Product-Revenue.txt	239 B
	6. Save Query Result to DBFS using Spark SQL.mp4	26.38 MB
	6. Save Query Result to DBFS using Spark SQL.vtt	6.31 KB
	6. Save-Query-Result-to-DBFS-using-Spark-SQL.txt	232 B
	7. Overview of Pyspark Examples on Databricks.mp4	3.9 MB
	7. Overview of Pyspark Examples on Databricks.vtt	1.44 KB
	7. Overview-of-Pyspark-Examples-on-Databricks.txt	229 B
	8. Process Schema Details in JSON using Pyspark.mp4	39.21 MB
	8. Process Schema Details in JSON using Pyspark.vtt	10.47 KB
	8. Process-Schema-Details-in-JSON-using-Pyspark.txt	233 B
	9. Create Dataframe with Schema from JSON File using Pyspark.mp4	22.11 MB
	9. Create Dataframe with Schema from JSON File using Pyspark.vtt	8.56 KB
	9. Create-Dataframe-with-Schema-from-JSON-File-using-Pyspark.txt	250 B
	23. Create Delta Tables using Spark SQL
	1. Introduction to Creating Delta Tables using Spark SQL.mp4	4.14 MB
	1. Introduction to Creating Delta Tables using Spark SQL.vtt	2.46 KB
	1. Introduction-to-Creating-Delta-Tables-using-Spark-SQL.txt	263 B
	10. Overview of Spark Metastore.mp4	24.44 MB
	10. Overview of Spark Metastore.vtt	7.1 KB
	11. Difference Between Managed and External Spark Metastore Tables.mp4	36.37 MB
	11. Difference Between Managed and External Spark Metastore Tables.vtt	10.11 KB
	11. Difference-Between-Managed-and-External-Spark-Metastore-Tables.txt	253 B
	12. Perform CRUD Operations on Delta Tables in Spark Metastore.mp4	20.34 MB
	12. Perform CRUD Operations on Delta Tables in Spark Metastore.vtt	8.77 KB
	12. Perform-CRUD-Operations-on-Delta-Tables-in-Spark-Metastore.txt	251 B
	13. Using Merge to Update and Insert into Delta Tables in Spark Metastore.mp4	44.39 MB
	13. Using Merge to Update and Insert into Delta Tables in Spark Metastore.vtt	13.58 KB
	13. Using-Merge-to-Update-and-Insert-into-Delta-Tables-in-Spark-Metastore.txt	268 B
	14. Conclusion of Creating Delta Tables using Spark SQL.mp4	5.94 MB
	14. Conclusion of Creating Delta Tables using Spark SQL.vtt	3.75 KB
	2. Overview of Supported Providers for Spark Metastore Tables.mp4	5.35 MB
	2. Overview of Supported Providers for Spark Metastore Tables.vtt	3.33 KB
	3. Create Database and Review the Details.mp4	30.82 MB
	3. Create Database and Review the Details.vtt	9.3 KB
	3. Create-Database-and-Review-the-Details.txt	225 B
	4. Create and Review Managed Spark Metastore Table using Delta Format.mp4	21.29 MB
	4. Create and Review Managed Spark Metastore Table using Delta Format.vtt	7.45 KB
	4. Create-and-Review-Managed-Spark-Metastore-Table-using-Delta-Format.txt	261 B
	5. Copy Data into Spark Metastore Managed Table.mp4	32.4 MB
	5. Copy Data into Spark Metastore Managed Table.vtt	7.33 KB
	5. Copy-Data-into-Spark-Metastore-Managed-Table.txt	233 B
	6. Validate Data in Spark Metastore Managed Table.mp4	12.75 MB
	6. Validate Data in Spark Metastore Managed Table.vtt	5.11 KB
	6. Validate-Data-in-Spark-Metastore-Managed-Table.txt	235 B
	7. Create and Review External Spark Metastore Table using Delta Format.mp4	19.43 MB
	7. Create and Review External Spark Metastore Table using Delta Format.vtt	6.55 KB
	7. Create-and-Review-External-Spark-Metastore-Table-using-Delta-Format.txt	262 B
	8. Insert Data into Spark Metastore External Table.mp4	19.4 MB
	8. Insert Data into Spark Metastore External Table.vtt	5.34 KB
	8. Insert-Data-into-Spark-Metastore-External-Table.txt	236 B
	9. Validate Data in Spark Metastore External Table.mp4	19.07 MB
	9. Validate Data in Spark Metastore External Table.vtt	4.57 KB
	9. Validate-Data-in-Spark-Metastore-External-Table.txt	236 B
	24. Pre-Defined Functions in Spark SQL
	1. Overview of Functions in Spark SQL.mp4	10.04 MB
	1. Overview of Functions in Spark SQL.vtt	4.3 KB
	1. Overview-of-Functions-in-Spark-SQL.txt	213 B
	10. Overview of Date Manipulation Functions in Spark SQL.mp4	4.21 MB
	10. Overview of Date Manipulation Functions in Spark SQL.vtt	2.39 KB
	10. Overview-of-Date-Manipulation-Functions-in-Spark-SQL.txt	219 B
	11. Overview of Standard Date and Timestamp in Spark SQL.mp4	17.35 MB
	11. Overview of Standard Date and Timestamp in Spark SQL.vtt	4.3 KB
	11. Overview-of-Standard-Date-and-Timestamp-in-Spark-SQL.txt	219 B
	12. Date Arithmetic using Spark SQL Functions.mp4	16.56 MB
	12. Date Arithmetic using Spark SQL Functions.vtt	6.01 KB
	12. Date-Arithmetic-using-Spark-SQL-Functions.txt	219 B
	13. Overview of trunc and date_trunc in Spark SQL.mp4	30.92 MB
	13. Overview of trunc and date_trunc in Spark SQL.vtt	6.72 KB
	13. Overview-of-trunc-and-date-trunc-in-Spark-SQL.txt	219 B
	14. Extract Information from Date or Time using Spark SQL.mp4	49.65 MB
	14. Extract Information from Date or Time using Spark SQL.vtt	10.59 KB
	14. Extract-Information-from-Date-or-Time-using-Spark-SQL.txt	219 B
	15. Convert Non Standard Dates or Timestamps to Standard Ones using Spark SQL.mp4	24.25 MB
	15. Convert Non Standard Dates or Timestamps to Standard Ones using Spark SQL.vtt	9.96 KB
	15. Convert-Non-Standard-Dates-or-Timestamps-to-Standard-Ones-using-Spark-SQL.txt	219 B
	16. Extract Information using Calendar Functions from Date or Timestamp using Spark.mp4	10.73 MB
	16. Extract Information using Calendar Functions from Date or Timestamp using Spark.vtt	3.85 KB
	16. Extract-Information-using-Calendar-Functions-from-Date-or-Timestamp-using-Spark.txt	219 B
	17. Dealing with Unix Timestamp using Spark SQL.mp4	33.3 MB
	17. Dealing with Unix Timestamp using Spark SQL.vtt	11.33 KB
	17. Dealing-with-Unix-Timestamp-using-Spark-SQL.txt	219 B
	18. Overview of Numeric Functions in Spark SQL.mp4	58.65 MB
	18. Overview of Numeric Functions in Spark SQL.vtt	15.61 KB
	18. Overview-of-Numeric-Functions-in-Spark-SQL.txt	221 B
	19. Data Type Conversion using Spark SQL.mp4	33.71 MB
	19. Data Type Conversion using Spark SQL.vtt	8.25 KB
	19. Data-Type-Conversion-using-Spark-SQL.txt	215 B
	2. Validate Functions in Spark SQL.mp4	14.49 MB
	2. Validate Functions in Spark SQL.vtt	5.48 KB
	2. Validate-Functions-in-Spark-SQL.txt	212 B
	20. Overview of Handling Null Values using Spark SQL.mp4	18.99 MB
	20. Overview of Handling Null Values using Spark SQL.vtt	5.06 KB
	20. Overview-of-Handling-Null-Values-using-Spark-SQL.txt	215 B
	21. Replace Null Values with default values using nvl and coalesce in Spark SQL.mp4	26.13 MB
	21. Replace Null Values with default values using nvl and coalesce in Spark SQL.vtt	8.46 KB
	21. Replace-Null-Values-with-default-values-using-nvl-and-coalesce-in-Spark-SQL.txt	215 B
	22. Conditional Logic on Null Values using nvl2 and case in Spark SQL.mp4	21.54 MB
	22. Conditional Logic on Null Values using nvl2 and case in Spark SQL.vtt	7.98 KB
	22. Conditional-Logic-on-Null-Values-using-nvl2-and-case-in-Spark-SQL.txt	215 B
	23. Overview of Case and When in Spark SQL.mp4	11.2 MB
	23. Overview of Case and When in Spark SQL.vtt	4.42 KB
	23. Overview-of-Case-and-When-in-Spark-SQL.txt	211 B
	24. Using CASE and WHEN for conditional logic in Spark SQL.mp4	33.44 MB
	24. Using CASE and WHEN for conditional logic in Spark SQL.vtt	7.29 KB
	24. Using-CASE-and-WHEN-for-conditional-logic-in-Spark-SQL.txt	211 B
	25. Aggregate using CASE and WHEN in GROUP BY in Spark SQL.mp4	18.04 MB
	25. Aggregate using CASE and WHEN in GROUP BY in Spark SQL.vtt	4.98 KB
	25. Aggregate-using-CASE-and-WHEN-in-GROUP-BY-in-Spark-SQL.txt	211 B
	26. Word Count Query using Pre-defined Functions in Spark SQL.mp4	38.09 MB
	26. Word Count Query using Pre-defined Functions in Spark SQL.vtt	8.56 KB
	26. Word-Count-Query-using-Pre-defined-Functions-in-Spark-SQL.txt	219 B
	27. Exercises for Pre-defined functions in Spark SQL.mp4	8.84 MB
	27. Exercises for Pre-defined functions in Spark SQL.vtt	2.76 KB
	27. Exercises-for-Pre-defined-functions-in-Spark-SQL.txt	223 B
	28. Solutions for Exercises 1 and 2 on Pre-defined Functions in Spark SQL.mp4	51.95 MB
	28. Solutions for Exercises 1 and 2 on Pre-defined Functions in Spark SQL.vtt	10.87 KB
	29. Solutions for Exercises 3 and 4 on Pre-defined Functions in Spark SQL.mp4	31.22 MB
	29. Solutions for Exercises 3 and 4 on Pre-defined Functions in Spark SQL.vtt	7.35 KB
	3. Overview of String Manipulation Functions in Spark SQL.mp4	4.89 MB
	3. Overview of String Manipulation Functions in Spark SQL.vtt	2.66 KB
	3. Overview-of-String-Manipulation-Functions-in-Spark-SQL.txt	221 B
	30. Solutions for Exercises 5 and 6 on Pre-defined Functions in Spark SQL.mp4	56.74 MB
	30. Solutions for Exercises 5 and 6 on Pre-defined Functions in Spark SQL.vtt	12.67 KB
	31. Solutions for Exercises 7 and 8 on Pre-defined Functions in Spark SQL.mp4	56.83 MB
	31. Solutions for Exercises 7 and 8 on Pre-defined Functions in Spark SQL.vtt	13.4 KB
	4. Case Conversion and Length of Strings using Spark SQL.mp4	23.52 MB
	4. Case Conversion and Length of Strings using Spark SQL.vtt	5.36 KB
	4. Case-Conversion-and-Length-of-Strings-using-Spark-SQL.txt	221 B
	5. Extract Substring using substr in Spark SQL.mp4	29.62 MB
	5. Extract Substring using substr in Spark SQL.vtt	6.51 KB
	5. Extract-Substring-using-substr-in-Spark-SQL.txt	221 B
	6. Extract Substrings from Delimited Strings using split in Spark SQL.mp4	16.87 MB
	6. Extract Substrings from Delimited Strings using split in Spark SQL.vtt	5.96 KB
	6. Extract-Substrings-from-Delimited-Strings-using-split-in-Spark-SQL.txt	221 B
	7. Trimming Characters or Strings using Spark SQL.mp4	30.47 MB
	7. Trimming Characters or Strings using Spark SQL.vtt	12.49 KB
	7. Trimming-Characters-or-Strings-using-Spark-SQL.txt	221 B
	8. Padding Characters to Strings using Spark SQL.mp4	35.18 MB
	8. Padding Characters to Strings using Spark SQL.vtt	10.12 KB
	8. Padding-Characters-to-Strings-using-Spark-SQL.txt	221 B
	9. Reverse and Concatenate Strings using Spark SQL.mp4	34.52 MB
	9. Reverse and Concatenate Strings using Spark SQL.vtt	9.18 KB
	9. Reverse-and-Concatenate-Strings-using-Spark-SQL.txt	221 B
	25. Setup Spark Metastore Tables for Basic Transformations
	1. Introduction to Basic Transformations using Spark SQL.mp4	4.12 MB
	1. Introduction to Basic Transformations using Spark SQL.vtt	2.77 KB
	2. Prepare Spark Metastore Tables for Basic Transformations.mp4	18.55 MB
	2. Prepare Spark Metastore Tables for Basic Transformations.vtt	6.35 KB
	3. Projecting Data using Spark SQL.mp4	22.31 MB
	3. Projecting Data using Spark SQL.vtt	8.54 KB
	26. Filtering Data using Spark SQL Queries
	1. Filtering Data using Equal Condition in Spark SQL.mp4	16.81 MB
	1. Filtering Data using Equal Condition in Spark SQL.vtt	7.1 KB
	1. Filtering-Data-using-Equal-Condition-in-Spark-SQL.txt	225 B
	2. Using IN, LIKE and BETWEEN in Spark SQL Queries.mp4	26.3 MB
	2. Using IN, LIKE and BETWEEN in Spark SQL Queries.vtt	6.55 KB
	2. Using-IN-LIKE-and-BETWEEN-in-Spark-SQL-Queries.txt	225 B
	3. Filter Data using Boolean AND in Spark SQL Queries.mp4	16.34 MB
	3. Filter Data using Boolean AND in Spark SQL Queries.vtt	6.36 KB
	3. Filter-Data-using-Boolean-AND-in-Spark-SQL-Queries.txt	225 B
	4. Filter Data using Boolean OR in Spark SQL Queries.mp4	13.5 MB
	4. Filter Data using Boolean OR in Spark SQL Queries.vtt	6.04 KB
	4. Filter-Data-using-Boolean-OR-in-Spark-SQL-Queries.txt	225 B
	5. Dealing with NULLS while Filtering Data in Spark SQL Queries.mp4	9.92 MB
	5. Dealing with NULLS while Filtering Data in Spark SQL Queries.vtt	4.98 KB
	5. Dealing-with-NULLS-while-Filtering-Data-in-Spark-SQL-Queries.txt	225 B
	27. Aggregations using Spark SQL Queries
	1. Perform Total Aggregations using Spark SQL Queries.mp4	33.78 MB
	1. Perform Total Aggregations using Spark SQL Queries.vtt	10.03 KB
	1. Perform-Total-Aggregations-using-Spark-SQL-Queries.txt	230 B
	2. Overview of Aggregations using GROUP BY in Spark SQL Queries.mp4	13.8 MB
	2. Overview of Aggregations using GROUP BY in Spark SQL Queries.vtt	6.45 KB
	2. Overview-of-Aggregations-using-GROUP-BY-in-Spark-SQL-Queries.txt	230 B
	3. GROUP BY Examples using Spark SQL Queries.mp4	14.24 MB
	3. GROUP BY Examples using Spark SQL Queries.vtt	7.15 KB
	3. GROUP-BY-Examples-using-Spark-SQL-Queries.txt	230 B
	4. Order of Execution of Spark SQL Queries.mp4	18.09 MB
	4. Order of Execution of Spark SQL Queries.vtt	9.12 KB
	4. Order-of-Execution-of-Spark-SQL-Queries.txt	230 B
	5. Filter Data based on Aggregate Results using HAVING in Spark SQL Queries.mp4	19.31 MB
	5. Filter Data based on Aggregate Results using HAVING in Spark SQL Queries.vtt	9.24 KB
	5. Filter-Data-based-on-Aggregate-Results-using-HAVING-in-Spark-SQL-Queries.txt	230 B
	28. Joins using Spark SQL Queries
	1. Overview of Joins in Spark SQL Queries.mp4	9.55 MB
	1. Overview of Joins in Spark SQL Queries.vtt	6.35 KB
	1. Overview-of-Joins-in-Spark-SQL-Queries.txt	216 B
	2. Inner Join using Spark SQL Queries.mp4	29.41 MB
	2. Inner Join using Spark SQL Queries.vtt	8.92 KB
	2. Inner-Join-using-Spark-SQL-Queries.txt	216 B
	3. Concepts Behind Inner Joins in Spark SQL.mp4	27.5 MB
	3. Concepts Behind Inner Joins in Spark SQL.vtt	7.2 KB
	3. Concepts-Behind-Inner-Joins-in-Spark-SQL.txt	216 B
	4. Outer Joins using Spark SQL Queries.mp4	42.84 MB
	4. Outer Joins using Spark SQL Queries.vtt	12.79 KB
	4. Outer-Joins-using-Spark-SQL-Queries.txt	216 B
	5. Example - Inner Join along with GROUP BY using Spark SQL Queries.mp4	12.87 MB
	5. Example - Inner Join along with GROUP BY using Spark SQL Queries.vtt	4.54 KB
	5. Example-Inner-Join-along-with-GROUP-BY-using-Spark-SQL-Queries.txt	216 B
	6. Example - Outer Join along with GROUP BY using Spark SQL Queries.mp4	13.53 MB
	6. Example - Outer Join along with GROUP BY using Spark SQL Queries.vtt	4.98 KB
	6. Example-Outer-Join-along-with-GROUP-BY-using-Spark-SQL-Queries.txt	216 B
	7. Example - Filtering and Outer Joins along with GROUP BY in Spark SQL Queries.mp4	20.67 MB
	7. Example - Filtering and Outer Joins along with GROUP BY in Spark SQL Queries.vtt	4.68 KB
	7. Example-Filtering-and-Outer-Joins-along-with-GROUP-BY-in-Spark-SQL-Queries.txt	216 B
	8. Example - Filtering and Outer Joins along with GROUP BY in Spark SQL Queries.mp4	15.44 MB
	8. Example - Filtering and Outer Joins along with GROUP BY in Spark SQL Queries.vtt	5.53 KB
	8. Example-Filtering-and-Outer-Joins-along-with-GROUP-BY-in-Spark-SQL-Queries.txt	216 B
	29. Sorting using Spark SQL Queries
	1. Sorting Data using Spark SQL Queries.mp4	15.5 MB
	1. Sorting Data using Spark SQL Queries.vtt	6.05 KB
	1. Sorting-Data-using-Spark-SQL-Queries.txt	207 B
	2. Dealing with Nulls while Sorting Data using Spark SQL Queries.mp4	15.06 MB
	2. Dealing with Nulls while Sorting Data using Spark SQL Queries.vtt	5.36 KB
	2. Dealing-with-Nulls-while-Sorting-Data-using-Spark-SQL-Queries.txt	207 B
	30. Copy Query Results into Spark Metastore Tables
	1. Overview of Copying Query Results into Spark Metastore Tables.mp4	4.9 MB
	1. Overview of Copying Query Results into Spark Metastore Tables.vtt	3.52 KB
	1. Overview-of-Copying-Query-Results-into-Spark-Metastore-Tables.txt	247 B
	2. Query to Compute Daily Revenue using Spark SQL.mp4	6.63 MB
	2. Query to Compute Daily Revenue using Spark SQL.vtt	3.15 KB
	2. Query-to-Compute-Daily-Revenue-using-Spark-SQL.txt	247 B
	3. Copy Query Results into Spark Metastore Tables using CTAS.mp4	16.62 MB
	3. Copy Query Results into Spark Metastore Tables using CTAS.vtt	6.92 KB
	3. Copy-Query-Results-into-Spark-Metastore-Tables-using-CTAS.txt	247 B
	4. Copy Query Results into Spark Metastore Tables using INSERT.mp4	13.5 MB
	4. Copy Query Results into Spark Metastore Tables using INSERT.vtt	5.59 KB
	4. Copy-Query-Results-into-Spark-Metastore-Tables-using-INSERT.txt	247 B
	5. Design Pipeline using CTAS and INSERT in Spark SQL.mp4	50.47 MB
	5. Design Pipeline using CTAS and INSERT in Spark SQL.vtt	13.2 KB
	5. Design-Pipeline-using-CTAS-and-INSERT-in-Spark-SQL.txt	247 B
	6. Copy Query Results into Spark Meatstore Tables using MERGE.mp4	25.44 MB
	6. Copy Query Results into Spark Meatstore Tables using MERGE.vtt	10.21 KB
	6. Copy-Query-Results-into-Spark-Meatstore-Tables-using-MERGE.txt	247 B
	31. Ranking using Spark SQL Windowing Functions
	1. Ranking using Spark SQL Windowing Functions.mp4	2.96 MB
	1. Ranking using Spark SQL Windowing Functions.vtt	2 KB
	2. Create Temporary View for ranking using Spark SQL Windowing Functions.mp4	5.53 MB
	2. Create Temporary View for ranking using Spark SQL Windowing Functions.vtt	2.49 KB
	3. Compute Global Rank using Spark SQL Windowing Functions.mp4	18.54 MB
	3. Compute Global Rank using Spark SQL Windowing Functions.vtt	7.84 KB
	4. Compute Ranks Per Key using Spark SQL Windowing Functions.mp4	14.92 MB
	4. Compute Ranks Per Key using Spark SQL Windowing Functions.vtt	5.32 KB
	5. Difference Between rank and dense_rank.mp4	13.18 MB
	5. Difference Between rank and dense_rank.vtt	5.92 KB
	6. Filter on Ranks using Spark SQL Windowing Functions.mp4	39.5 MB
	6. Filter on Ranks using Spark SQL Windowing Functions.vtt	9.52 KB
	32. Processing JSON like Data using Spark SQL
	1. Overview of JSON.mp4	10.63 MB
	1. Overview of JSON.vtt	6.94 KB
	1. Overview-of-JSON.txt	235 B
	10. Generate Array of Struct Type Columns from Regular Columns in Spark SQL.mp4	19.92 MB
	10. Generate Array of Struct Type Columns from Regular Columns in Spark SQL.vtt	5.94 KB
	10. Generate-Array-of-Struct-Type-Columns-from-Regular-Columns-in-Spark-SQL.txt	235 B
	11. Processing Delimited Strings using Spark SQL Queries.mp4	19.09 MB
	11. Processing Delimited Strings using Spark SQL Queries.vtt	6.3 KB
	11. Processing-Delimited-Strings-using-Spark-SQL-Queries.txt	235 B
	2. Creating Spark Metastore Tables with Array Type Columns.mp4	12.06 MB
	2. Creating Spark Metastore Tables with Array Type Columns.vtt	6.36 KB
	2. Creating-Spark-Metastore-Tables-with-Array-Type-Columns.txt	235 B
	3. Dealing with Array Type Columns using Spark SQL Queries.mp4	20.81 MB
	3. Dealing with Array Type Columns using Spark SQL Queries.vtt	7.39 KB
	3. Dealing-with-Array-Type-Columns-using-Spark-SQL-Queries.txt	235 B
	4. Creating Spark Metastore Tables with Struct Type Columns.mp4	15.35 MB
	4. Creating Spark Metastore Tables with Struct Type Columns.vtt	4.96 KB
	4. Creating-Spark-Metastore-Tables-with-Struct-Type-Columns.txt	235 B
	5. Projecting Data From Struct Type Fields in Spark SQL.mp4	18.26 MB
	5. Projecting Data From Struct Type Fields in Spark SQL.vtt	6.14 KB
	5. Projecting-Data-From-Struct-Type-Fields-in-Spark-SQL.txt	235 B
	6. Creating Spark Metastore Tables with Array of Struct Column.mp4	17.42 MB
	6. Creating Spark Metastore Tables with Array of Struct Column.vtt	5.65 KB
	6. Creating-Spark-Metastore-Tables-with-Array-of-Struct-Column.txt	235 B
	7. Dealing with Array of Struct Type Columns using Spark SQL Queries.mp4	21.02 MB
	7. Dealing with Array of Struct Type Columns using Spark SQL Queries.vtt	6.58 KB
	7. Dealing-with-Array-of-Struct-Type-Columns-using-Spark-SQL-Queries.txt	235 B
	8. Overview of Important Functions to Process JSON Data in Spark SQL.mp4	11.98 MB
	8. Overview of Important Functions to Process JSON Data in Spark SQL.vtt	5.45 KB
	8. Overview-of-Important-Functions-to-Process-JSON-Data-in-Spark-SQL.txt	235 B
	9. Generate Array Type Columns from Regular Columns in Spark SQL.mp4	20.89 MB
	9. Generate Array Type Columns from Regular Columns in Spark SQL.vtt	7.02 KB
	9. Generate-Array-Type-Columns-from-Regular-Columns-in-Spark-SQL.txt	235 B
	33. Getting Started with Pyspark Data Frame APIs
	1. Overview of Pyspark Examples on Databricks.mp4	3.91 MB
	1. Overview of Pyspark Examples on Databricks.vtt	1.44 KB
	2. Process Schema Details in JSON using Pyspark.mp4	39.2 MB
	2. Process Schema Details in JSON using Pyspark.vtt	10.45 KB
	3. Create Dataframe with Schema from JSON File using Pyspark.mp4	43.16 MB
	3. Create Dataframe with Schema from JSON File using Pyspark.vtt	8.56 KB
	4. Transform Data using Spark APIs.mp4	22.41 MB
	4. Transform Data using Spark APIs.vtt	6.05 KB
	5. Get Schema Details for all Data Sets using Pyspark.mp4	13.05 MB
	5. Get Schema Details for all Data Sets using Pyspark.vtt	5.61 KB
	6. Convert CSV to Parquet with Schema using Pyspark.mp4	29.7 MB
	6. Convert CSV to Parquet with Schema using Pyspark.vtt	6.81 KB
	34. Create Spark Data Frames using Pyspark Data Frame APIs
	1. Create Spark Data Frame using Pyspark Data Frame APIs.mp4	13.64 MB
	1. Create Spark Data Frame using Pyspark Data Frame APIs.vtt	6.38 KB
	10. Using withColumn to apply transformations on Spark Data Frames.mp4	11.06 MB
	10. Using withColumn to apply transformations on Spark Data Frames.vtt	5.06 KB
	11. Overview of Writing Data in Data Frame to Delta Files.mp4	25.56 MB
	11. Overview of Writing Data in Data Frame to Delta Files.vtt	7.17 KB
	2. Introduction to Processing JSON like Data using Spark SQL.mp4	2.34 MB
	2. Introduction to Processing JSON like Data using Spark SQL.vtt	1.8 KB
	3. Overview of Data Processing using Conventional loops.mp4	30.09 MB
	3. Overview of Data Processing using Conventional loops.vtt	9.65 KB
	4. Overview of Data Frame Concepts.mp4	16.9 MB
	4. Overview of Data Frame Concepts.vtt	8.19 KB
	5. Advantages of Pyspark Data Frames.mp4	9.19 MB
	5. Advantages of Pyspark Data Frames.vtt	4.77 KB
	6. Overview of Spark Data Frames and their Characteristics.mp4	19.22 MB
	6. Overview of Spark Data Frames and their Characteristics.vtt	7.65 KB
	7. Projecting Data in Spark Data Frames using Select.mp4	9.48 MB
	7. Projecting Data in Spark Data Frames using Select.vtt	5.77 KB
	8. Using drop to drop columns from Spark Data Frame.mp4	7.52 MB
	8. Using drop to drop columns from Spark Data Frame.vtt	3.71 KB
	9. Applying functions on Spark Data Frame Columns.mp4	18.67 MB
	9. Applying functions on Spark Data Frame Columns.vtt	8.79 KB
	35. Basic Transformations using Pyspark Data Frame APIs
	1. Overview of Basic Transformations using Pyspark Data Frame APIs.mp4	5.57 MB
	1. Overview of Basic Transformations using Pyspark Data Frame APIs.vtt	4.46 KB
	1. Overview-of-Basic-Transformations-using-Pyspark-Data-Frame-APIs.txt	275 B
	10. Composite Sorting using Spark Data Frame APIs.mp4	35.53 MB
	10. Composite Sorting using Spark Data Frame APIs.vtt	9.22 KB
	10. Composite-Sorting-using-Spark-Data-Frame-APIs.txt	275 B
	11. Develop Spark SQL Queries for Sorting Data.mp4	10.41 MB
	11. Develop Spark SQL Queries for Sorting Data.vtt	4.24 KB
	11. Develop-Spark-SQL-Queries-for-Sorting-Data.txt	275 B
	12. Review Data Set with Nulls for Sorting using Spark Data Frame APIs.mp4	34.89 MB
	12. Review Data Set with Nulls for Sorting using Spark Data Frame APIs.vtt	7.81 KB
	12. Review-Data-Set-with-Nulls-for-Sorting-using-Spark-Data-Frame-APIs.txt	275 B
	13. Dealing with Nulls while Sorting the Data in Spark Data Frames.mp4	23.51 MB
	13. Dealing with Nulls while Sorting the Data in Spark Data Frames.vtt	6.25 KB
	13. Dealing-with-Nulls-while-Sorting-the-Data-in-Spark-Data-Frames.txt	275 B
	2. Overview of Row Level Transformations.mp4	13.07 MB
	2. Overview of Row Level Transformations.vtt	7.73 KB
	2. Overview-of-Row-Level-Transformations.txt	275 B
	3. Apply Row Level transformations using Pyspark Data Frame APIs.mp4	28.3 MB
	3. Apply Row Level transformations using Pyspark Data Frame APIs.vtt	11.43 KB
	3. Apply-Row-Level-transformations-using-Pyspark-Data-Frame-APIs.txt	275 B
	4. Filtering Data using Pyspark Data Frame APIs.mp4	20 MB
	4. Filtering Data using Pyspark Data Frame APIs.vtt	6.99 KB
	4. Filtering-Data-using-Pyspark-Data-Frame-APIs.txt	275 B
	5. Filtering Data with Multiple Conditions using Pyspark Data Frame APIs.mp4	26.09 MB
	5. Filtering Data with Multiple Conditions using Pyspark Data Frame APIs.vtt	8.41 KB
	5. Filtering-Data-with-Multiple-Conditions-using-Pyspark-Data-Frame-APIs.txt	275 B
	6. Perform Aggregations by Key using Spark Data Frame APIs.mp4	19.72 MB
	6. Perform Aggregations by Key using Spark Data Frame APIs.vtt	6.63 KB
	6. Perform-Aggregations-by-Key-using-Spark-Data-Frame-APIs.txt	275 B
	7. Perform Aggregations by Key using Spark Data Frame APIs.mp4	23.21 MB
	7. Perform Aggregations by Key using Spark Data Frame APIs.vtt	9.39 KB
	7. Perform-Aggregations-by-Key-using-Spark-Data-Frame-APIs.txt	275 B
	8. Perform Aggregations by Key using Spark Data Frame APIs.mp4	9.92 MB
	8. Perform Aggregations by Key using Spark Data Frame APIs.vtt	6.04 KB
	8. Perform-Aggregations-by-Key-using-Spark-Data-Frame-APIs.txt	275 B
	9. Sorting Data using Spark Data Frame APIs.mp4	17.05 MB
	9. Sorting Data using Spark Data Frame APIs.vtt	8.5 KB
	9. Sorting-Data-using-Spark-Data-Frame-APIs.txt	275 B
	36. Joining Data using Spark Data Frame APIs
	1. Introduction to Joining Data using Spark Data Frame APIs.mp4	2.69 MB
	1. Introduction to Joining Data using Spark Data Frame APIs.vtt	2.43 KB
	1. Introduction-to-Joining-Data-using-Spark-Data-Frame-APIs.txt	253 B
	2. Create Data Frames to Join using Spark Data Frame APIs.mp4	23.68 MB
	2. Create Data Frames to Join using Spark Data Frame APIs.vtt	7.57 KB
	2. Create-Data-Frames-to-Join-using-Spark-Data-Frame-APIs.txt	253 B
	3. Review Syntax for join using Spark Data Frame APIs.mp4	7.02 MB
	3. Review Syntax for join using Spark Data Frame APIs.vtt	4.51 KB
	3. Review-Syntax-for-join-using-Spark-Data-Frame-APIs.txt	253 B
	4. Inner Join using Spark Data Frame APIs.mp4	19.49 MB
	4. Inner Join using Spark Data Frame APIs.vtt	8.22 KB
	4. Inner-Join-using-Spark-Data-Frame-APIs.txt	253 B
	5. Join and other Spark Data Frame APIs to process the data.mp4	26.36 MB
	5. Join and other Spark Data Frame APIs to process the data.vtt	9.09 KB
	5. Join-and-other-Spark-Data-Frame-APIs-to-process-the-data.txt	253 B
	6. Analyze Data for outer joins using Spark Data Frame APIs.mp4	9.71 MB
	6. Analyze Data for outer joins using Spark Data Frame APIs.vtt	4.52 KB
	6. Analyze-Data-for-outer-joins-using-Spark-Data-Frame-APIs.txt	253 B
	7. Left Outer Join using Spark Data Frame APIs.mp4	17.71 MB
	7. Left Outer Join using Spark Data Frame APIs.vtt	9.04 KB
	7. Left-Outer-Join-using-Spark-Data-Frame-APIs.txt	253 B
	8. Right Outer Join using Spark Data Frame APIs.mp4	12.65 MB
	8. Right Outer Join using Spark Data Frame APIs.vtt	6.42 KB
	8. Right-Outer-Join-using-Spark-Data-Frame-APIs.txt	253 B
	9. Equivalent Spark SQL Queries for Joins.mp4	25.19 MB
	9. Equivalent Spark SQL Queries for Joins.vtt	7.38 KB
	9. Equivalent-Spark-SQL-Queries-for-Joins.txt	253 B
	37. Ranking using Pyspark Data Frame APIs
	1. Introduction to Ranking using Spark Data Frame APIs.mp4	26.63 MB
	1. Introduction to Ranking using Spark Data Frame APIs.vtt	8.87 KB
	1. Introduction-to-Ranking-using-Spark-Data-Frame-APIs.txt	243 B
	2. Syntax for ranking using Spark Data Frame APIs.mp4	5.68 MB
	2. Syntax for ranking using Spark Data Frame APIs.vtt	4.67 KB
	2. Syntax-for-ranking-using-Spark-Data-Frame-APIs.txt	243 B
	3. Compute Global Ranks using Spark Data Frame APIs.mp4	15.92 MB
	3. Compute Global Ranks using Spark Data Frame APIs.vtt	7.7 KB
	3. Compute-Global-Ranks-using-Spark-Data-Frame-APIs.txt	243 B
	4. Filter Based on Global Ranks using Spark Data Frame APIs.mp4	21.32 MB
	4. Filter Based on Global Ranks using Spark Data Frame APIs.vtt	6.31 KB
	4. Filter-Based-on-Global-Ranks-using-Spark-Data-Frame-APIs.txt	243 B
	5. Compute Ranks per Partition using Spark Data Frame APIs.mp4	10.6 MB
	5. Compute Ranks per Partition using Spark Data Frame APIs.vtt	5.32 KB
	5. Compute-Ranks-per-Partition-using-Spark-Data-Frame-APIs.txt	243 B
	6. Filter Based on Ranks Per Partition using Spark Data Frame APIs.mp4	14.27 MB
	6. Filter Based on Ranks Per Partition using Spark Data Frame APIs.vtt	5.53 KB
	6. Filter-Based-on-Ranks-Per-Partition-using-Spark-Data-Frame-APIs.txt	243 B
	7. Difference Between rank and dense_rank.mp4	26.71 MB
	7. Difference Between rank and dense_rank.vtt	11.82 KB
	7. Difference-Between-rank-and-dense-rank.txt	243 B
	38. Integration of Spark SQL and Pyspark Data Frame APIs
	1. Introduction to Integration of Spark SQL and Pyspark Data Frame APIs.mp4	4.02 MB
	1. Introduction to Integration of Spark SQL and Pyspark Data Frame APIs.vtt	2.6 KB
	2. Run Spark SQL Queries on Spark Data Frames.mp4	26.73 MB
	2. Run Spark SQL Queries on Spark Data Frames.vtt	7.84 KB
	3. Create Spark Metastore Tables using Data Frames.mp4	18.79 MB
	3. Create Spark Metastore Tables using Data Frames.vtt	7.23 KB
	4. Insert into Spark Metastore Tables using Data Frames.mp4	14.98 MB
	4. Insert into Spark Metastore Tables using Data Frames.vtt	6.28 KB
	5. Read Data from Spark Metastore Table to Data Frames.mp4	12.77 MB
	5. Read Data from Spark Metastore Table to Data Frames.vtt	5.9 KB
	6. Process Data in Spark Metastore Tables using Data Frame APIs.mp4	26.66 MB
	6. Process Data in Spark Metastore Tables using Data Frame APIs.vtt	7.59 KB
	7. Manage Spark Metastore Database Objects using Spark APIs.mp4	29.65 MB
	7. Manage Spark Metastore Database Objects using Spark APIs.vtt	7.8 KB
	39. ELT Data Pipelines using Databricks
	1. Overview of Databricks Workflows.mp4	10.98 MB
	1. Overview of Databricks Workflows.vtt	4.56 KB
	10. Review Databricks SQL Notebooks for Tables and Final Results.mp4	9.83 MB
	10. Review Databricks SQL Notebooks for Tables and Final Results.vtt	5.81 KB
	11. Validate Applications for ELT Pipeline using Databricks.mp4	43.24 MB
	11. Validate Applications for ELT Pipeline using Databricks.vtt	11.39 KB
	12. Build ELT Pipeline using Databricks Job in Workflows.mp4	28.94 MB
	12. Build ELT Pipeline using Databricks Job in Workflows.vtt	13.33 KB
	13. Run and Review Execution details of ELT Data Pipeline using Databricks Job.mp4	15.95 MB
	13. Run and Review Execution details of ELT Data Pipeline using Databricks Job.vtt	7.38 KB
	2. Pass Arguments to Databricks Python Notebooks.mp4	22.15 MB
	2. Pass Arguments to Databricks Python Notebooks.vtt	10.01 KB
	3. Pass Arguments to Databricks SQL Notebooks.mp4	13.27 MB
	3. Pass Arguments to Databricks SQL Notebooks.vtt	4.32 KB
	4. Create and Run First Databricks Job.mp4	16.91 MB
	4. Create and Run First Databricks Job.vtt	10.84 KB
	5. Run Databricks Jobs and Tasks with Parameters.mp4	28.44 MB
	5. Run Databricks Jobs and Tasks with Parameters.vtt	7.49 KB
	6. Create and Run Orchestrated Pipeline using Databricks Job.mp4	34.64 MB
	6. Create and Run Orchestrated Pipeline using Databricks Job.vtt	9.69 KB
	7. Import ELT Data Pipeline Applications into Databricks Environment.mp4	9.83 MB
	7. Import ELT Data Pipeline Applications into Databricks Environment.vtt	4.16 KB
	7. Import-ELT-Data-Pipeline-Applications-into-Databricks-Environment.txt	133 B
	8. Spark SQL Application to Cleanup Database and Datasets.mp4	8.85 MB
	8. Spark SQL Application to Cleanup Database and Datasets.vtt	5.45 KB
	9. Review File Format Converter Pyspark Code.mp4	30.1 MB
	9. Review File Format Converter Pyspark Code.vtt	7.46 KB
	40. Performance Tuning of Spark - Catalyst Optimizer
	1. Getting Started with Performance Tuning using Spark on Databricks.mp4	7.19 MB
	1. Getting Started with Performance Tuning using Spark on Databricks.vtt	4.58 KB
	10. Understand Join and Aggregation for Daily Product Revenue.mp4	30.07 MB
	10. Understand Join and Aggregation for Daily Product Revenue.vtt	6.62 KB
	2. Overview of Spark Catalyst Optimizer.mp4	11.92 MB
	2. Overview of Spark Catalyst Optimizer.vtt	5.88 KB
	3. Review Explain Plan for Spark Dataframe logic using Spark UI.mp4	21.05 MB
	3. Review Explain Plan for Spark Dataframe logic using Spark UI.vtt	5.61 KB
	4. Review Explain Plan for Spark SQL logic using Spark UI.mp4	18.27 MB
	4. Review Explain Plan for Spark SQL logic using Spark UI.vtt	4.66 KB
	5. Generate Explain Plans on Spark Dataframes using explain function.mp4	25.75 MB
	5. Generate Explain Plans on Spark Dataframes using explain function.vtt	5.74 KB
	6. Generate Explain Plans on Spark SQL Queries using explain command.mp4	19.07 MB
	6. Generate Explain Plans on Spark SQL Queries using explain command.vtt	3.79 KB
	7. Interpreting Explain Plan for Spark SQL Query.mp4	26.01 MB
	7. Interpreting Explain Plan for Spark SQL Query.vtt	7.45 KB
	8. Overview of Spark Architecture.mp4	16.13 MB
	8. Overview of Spark Architecture.vtt	5.17 KB
	9. Understand Filter and Broadcast of Orders Data.mp4	20.91 MB
	9. Understand Filter and Broadcast of Orders Data.vtt	8.03 KB
	41. Performance Tuning of Spark - Cluster Configuration
	1. Introduction to Databricks Cluster Configuration.mp4	5.02 MB
	1. Introduction to Databricks Cluster Configuration.vtt	2.76 KB
	10. Review Running Job Details using Spark UI.mp4	21.58 MB
	10. Review Running Job Details using Spark UI.vtt	4.58 KB
	11. Review Completed Job Details using Spark UI.mp4	13.01 MB
	11. Review Completed Job Details using Spark UI.vtt	5.25 KB
	2. Difference Between All Purpose and Jobs Clusters.mp4	27.07 MB
	2. Difference Between All Purpose and Jobs Clusters.vtt	7.41 KB
	3. Setting up All Purpose Databricks Compute Clusters.mp4	11.29 MB
	3. Setting up All Purpose Databricks Compute Clusters.vtt	7.25 KB
	4. Understand the size of the data using dbutils.mp4	32.99 MB
	4. Understand the size of the data using dbutils.vtt	8.86 KB
	5. Create Multinode Databricks Cluster with Auto Scaling.mp4	10.01 MB
	5. Create Multinode Databricks Cluster with Auto Scaling.vtt	4.56 KB
	6. Overview of Auto Scaling of Databricks Clusters.mp4	12.47 MB
	6. Overview of Auto Scaling of Databricks Clusters.vtt	4.4 KB
	7. Performance Tuning of Cluster using Auto Scaling.mp4	11.9 MB
	7. Performance Tuning of Cluster using Auto Scaling.vtt	5.94 KB
	8. Analyze Airlines Data using Spark SQL.mp4	16.66 MB
	8. Analyze Airlines Data using Spark SQL.vtt	4.49 KB
	9. Setup Databricks Job Compute Clusters using Workflows.mp4	21.1 MB
	9. Setup Databricks Job Compute Clusters using Workflows.vtt	7.06 KB
	42. Performance Tuning while inferring schema from CSV or JSON files
	1. Overview of Inferring Schema using CSV or JSON Files.mp4	4.49 MB
	1. Overview of Inferring Schema using CSV or JSON Files.vtt	2.51 KB
	2. Steps to convert CSV or JSON Files to Parquet or Delta Files.mp4	9.92 MB
	2. Steps to convert CSV or JSON Files to Parquet or Delta Files.vtt	4.84 KB
	3. Overview of CSV or JSON Files.mp4	17.25 MB
	3. Overview of CSV or JSON Files.vtt	6.89 KB
	4. Overview of overhead for inferring schema.mp4	18.27 MB
	4. Overview of overhead for inferring schema.vtt	8.75 KB
	5. Performance Tuning to infer schema of Spark Dataframe.mp4	13.29 MB
	5. Performance Tuning to infer schema of Spark Dataframe.vtt	7.67 KB
	43. Performance Tuning using Columnar File Format and Partitioning Strategy
	1. Introduction to Performance Tuning while storing data in Data Lake.mp4	3.5 MB
	1. Introduction to Performance Tuning while storing data in Data Lake.vtt	1.92 KB
	10. Overview of Folder Structure for Partitioned Data in Spark.mp4	27.09 MB
	10. Overview of Folder Structure for Partitioned Data in Spark.vtt	6.18 KB
	11. Evaluate Requirements against Partition Pruning.mp4	37.64 MB
	11. Evaluate Requirements against Partition Pruning.vtt	8.08 KB
	12. Solutions on Airlines Data for Performance Tuning using Partitioning.mp4	23.78 MB
	12. Solutions on Airlines Data for Performance Tuning using Partitioning.vtt	8.86 KB
	13. Review Execution Details for Performance Tuning using Partitioning.mp4	73.22 MB
	13. Review Execution Details for Performance Tuning using Partitioning.vtt	13.78 KB
	14. Get Airlines Data using Date Range without Partition Pruning.mp4	22.8 MB
	14. Get Airlines Data using Date Range without Partition Pruning.vtt	8.65 KB
	15. Parameterize Spark SQL Solution for Partition Pruning.mp4	21.27 MB
	15. Parameterize Spark SQL Solution for Partition Pruning.vtt	5 KB
	16. Add Condition for Partition Pruning.mp4	31.06 MB
	16. Add Condition for Partition Pruning.vtt	6.73 KB
	17. Redesign Partition Strategy to tune the performance.mp4	37.6 MB
	17. Redesign Partition Strategy to tune the performance.vtt	7.77 KB
	18. Recap of Spark Performance Tuning Scenarios.mp4	14.45 MB
	18. Recap of Spark Performance Tuning Scenarios.vtt	6.53 KB
	2. Side effects of using CSV Files in Data Lake.mp4	27.61 MB
	2. Side effects of using CSV Files in Data Lake.vtt	6.38 KB
	3. Review the side effects of using CSV Files in Data Lake.mp4	11.31 MB
	3. Review the side effects of using CSV Files in Data Lake.vtt	4.33 KB
	4. Restructure CSV Data to Columnar Format using Pyspark.mp4	25.85 MB
	4. Restructure CSV Data to Columnar Format using Pyspark.vtt	6.86 KB
	5. Compute Size of restrucutured data using Parquet File Format.mp4	24.68 MB
	5. Compute Size of restrucutured data using Parquet File Format.vtt	5.38 KB
	6. Run Operations on Partitioned Parquet Data.mp4	14.13 MB
	6. Run Operations on Partitioned Parquet Data.vtt	6.25 KB
	7. Overview of Performance Assessment of Spark Jobs.mp4	24.57 MB
	7. Overview of Performance Assessment of Spark Jobs.vtt	5.43 KB
	8. Review Performance Details of Spark Operations on Parquet Files.mp4	36.9 MB
	8. Review Performance Details of Spark Operations on Parquet Files.vtt	10.7 KB
	9. Overview of Columnar File Formats in Spark and Databricks.mp4	24.3 MB
	9. Overview of Columnar File Formats in Spark and Databricks.vtt	9 KB
	44. Setup Hadoop and Spark Cluster using Dataproc
	1. Introduction to Setup Hadoop and Spark Cluster using Dataproc.mp4	6.6 MB
	1. Introduction to Setup Hadoop and Spark Cluster using Dataproc.vtt	4.74 KB
	10. Getting Started with Spark CLI using Python.mp4	27.01 MB
	10. Getting Started with Spark CLI using Python.vtt	6.36 KB
	11. Getting Started with Spark CLI using Scala.mp4	18.75 MB
	11. Getting Started with Spark CLI using Scala.vtt	5.27 KB
	12. Getting Started with Spark CLI using SQL.mp4	11.34 MB
	12. Getting Started with Spark CLI using SQL.vtt	7.3 KB
	13. Stopping the Cluster and Understanding the costs.mp4	14.46 MB
	13. Stopping the Cluster and Understanding the costs.vtt	7.93 KB
	14. Download VS Code Workspace and Delete Cluster.mp4	17.66 MB
	14. Download VS Code Workspace and Delete Cluster.vtt	6.52 KB
	2. Overview of different Spark Platforms on Cloud.mp4	24.4 MB
	2. Overview of different Spark Platforms on Cloud.vtt	8.07 KB
	3. Overview of Hadoop and Spark Cluster Types and Architecture.mp4	44.85 MB
	3. Overview of Hadoop and Spark Cluster Types and Architecture.vtt	13.34 KB
	4. Setup Single Node Hadoop and Spark Cluster using Dataproc.mp4	14.02 MB
	4. Setup Single Node Hadoop and Spark Cluster using Dataproc.vtt	5.09 KB
	5. Validate SSH Connectivity to the Dataproc Cluster.mp4	31.65 MB
	5. Validate SSH Connectivity to the Dataproc Cluster.vtt	8.9 KB
	6. Convert IP Address to Static for Dataproc Cluster.mp4	19.65 MB
	6. Convert IP Address to Static for Dataproc Cluster.vtt	5.43 KB
	7. Setup Project using VS Code Remote Development.mp4	93.86 MB
	7. Setup Project using VS Code Remote Development.vtt	20.1 KB
	8. Setup Local Data Sets on Hadoop and Spark Cluster.mp4	7.74 MB
	8. Setup Local Data Sets on Hadoop and Spark Cluster.vtt	3.15 KB
	9. Getting Started with HDFS Commands to Manage Files.mp4	56.54 MB
	9. Getting Started with HDFS Commands to Manage Files.vtt	10.34 KB
	45. Recap of important Linux Commands for Data Engineering
	1. Introduction to Linux Commands and Scripts for Data Engineers.mp4	5.44 MB
	1. Introduction to Linux Commands and Scripts for Data Engineers.vtt	3.72 KB
	10. Listing Files and Folders using ls command.mp4	80.13 MB
	10. Listing Files and Folders using ls command.vtt	13.59 KB
	11. Searching for files using find command in Linux.mp4	22.38 MB
	11. Searching for files using find command in Linux.vtt	8.25 KB
	12. Standard Directories in Linux.mp4	64.43 MB
	12. Standard Directories in Linux.vtt	11.8 KB
	13. Troubleshooting issues in Linux using grep command.mp4	157.66 MB
	13. Troubleshooting issues in Linux using grep command.vtt	17.47 KB
	14. Overview of Shell Scripts.mp4	61.78 MB
	14. Overview of Shell Scripts.vtt	14.27 KB
	15. Running and Debugging Shell Scripts with Arguments.mp4	21.65 MB
	15. Running and Debugging Shell Scripts with Arguments.vtt	13.01 KB
	16. Overview of Hadoop and Spark Executables.mp4	69.58 MB
	16. Overview of Hadoop and Spark Executables.vtt	10.4 KB
	2. Overview of SSH to connect to remote Servers.mp4	113.09 MB
	2. Overview of SSH to connect to remote Servers.vtt	27.51 KB
	3. Overview of Profile in Linux Shell.mp4	36.71 MB
	3. Overview of Profile in Linux Shell.vtt	10.17 KB
	4. Overview of Environment Variables in Linux.mp4	42.61 MB
	4. Overview of Environment Variables in Linux.vtt	9.13 KB
	5. Understanding PATH Environment Variable.mp4	132.78 MB
	5. Understanding PATH Environment Variable.vtt	24.14 KB
	6. Creating Folders in Linux using mkdir.mp4	31.8 MB
	6. Creating Folders in Linux using mkdir.vtt	12.26 KB
	7. Copy Files and Folders in Linux using cp command.mp4	75.64 MB
	7. Copy Files and Folders in Linux using cp command.vtt	13.22 KB
	8. Move Files and Folders in Linux using mv command.mp4	27.56 MB
	8. Move Files and Folders in Linux using mv command.vtt	9.02 KB
	9. Delete Files and Folders in Linux using rm command.mp4	61.64 MB
	9. Delete Files and Folders in Linux using rm command.vtt	12 KB
	46. Mastering Hadoop HDFS Commands and Concepts
	1. Introduction to Mastering Hadoop HDFS Commands and Concepts.mp4	3.17 MB
	1. Introduction to Mastering Hadoop HDFS Commands and Concepts.vtt	3.13 KB
	10. Overview of local storage of files.mp4	57.62 MB
	10. Overview of local storage of files.vtt	12.07 KB
	11. Setup Data Sets to understand HDFS Concepts.mp4	59.09 MB
	11. Setup Data Sets to understand HDFS Concepts.vtt	7.61 KB
	12. Overview of Distributed Storage of files in HDFS.mp4	29.08 MB
	12. Overview of Distributed Storage of files in HDFS.vtt	8.21 KB
	13. Determining Number of Blocks for each file.mp4	58.38 MB
	13. Determining Number of Blocks for each file.vtt	9.71 KB
	14. Overview of Blocks related to files in HDFS.mp4	83.75 MB
	14. Overview of Blocks related to files in HDFS.vtt	11.17 KB
	15. Overview of Replication related to files in HDFS.mp4	61.98 MB
	15. Overview of Replication related to files in HDFS.vtt	9.99 KB
	16. Physical Storage of HDFS File Blocks.mp4	55.02 MB
	16. Physical Storage of HDFS File Blocks.vtt	8.5 KB
	17. Overview of HDFS Namenode for HDFS File Metadata.mp4	64.61 MB
	17. Overview of HDFS Namenode for HDFS File Metadata.vtt	10.22 KB
	18. Recap of HDFS on Dataproc Cluster.mp4	86.32 MB
	18. Recap of HDFS on Dataproc Cluster.vtt	14.48 KB
	2. Start the Hadoop and Spark Cluster using Dataproc.mp4	5.9 MB
	2. Start the Hadoop and Spark Cluster using Dataproc.vtt	3.3 KB
	3. Create Hadoop and Spark Cluster and Setup VS Code Workspace.mp4	38.52 MB
	3. Create Hadoop and Spark Cluster and Setup VS Code Workspace.vtt	9.82 KB
	4. Overview of important HDFS Commands.mp4	16.15 MB
	4. Overview of important HDFS Commands.vtt	3.68 KB
	5. Create Folder and Copy Files into HDFS using Commands.mp4	33.87 MB
	5. Create Folder and Copy Files into HDFS using Commands.vtt	11.25 KB
	6. Programmatically Copy files into HDFS using Python.mp4	64.23 MB
	6. Programmatically Copy files into HDFS using Python.vtt	13.47 KB
	7. Overview of Multinode Hadoop Cluster.mp4	63.67 MB
	7. Overview of Multinode Hadoop Cluster.vtt	13.79 KB
	8. Review Important Properties of HDFS.mp4	37.35 MB
	8. Review Important Properties of HDFS.vtt	8.2 KB
	9. Review HDFS Properties on Dataproc Cluster using VS Code.mp4	29.84 MB
	9. Review HDFS Properties on Dataproc Cluster using VS Code.vtt	5.5 KB
	47. Build Hive Applications in Hadoop and Spark Clusters
	1. Introduction to Building Hive Applications.mp4	2.81 MB
	1. Introduction to Building Hive Applications.vtt	2.77 KB
	10. Populate Data into Partitioned NYSE table from Stage Table.mp4	28.74 MB
	10. Populate Data into Partitioned NYSE table from Stage Table.vtt	13.92 KB
	11. Process NYSE Data and load into partitioned table.mp4	37.85 MB
	11. Process NYSE Data and load into partitioned table.vtt	7.59 KB
	12. Develop Hive QL Script to Load NYSE Data.mp4	33.55 MB
	12. Develop Hive QL Script to Load NYSE Data.vtt	7.04 KB
	13. Run Hive QL Commands using Script for NYSE Loader.mp4	52.43 MB
	13. Run Hive QL Commands using Script for NYSE Loader.vtt	10.82 KB
	14. Redesign the Solution using HDFS to stage files.mp4	43.04 MB
	14. Redesign the Solution using HDFS to stage files.vtt	15.91 KB
	15. Validate Hive Application to Convert NYSE Data.mp4	49.93 MB
	15. Validate Hive Application to Convert NYSE Data.vtt	11.52 KB
	16. Deploy Hive Application in HDFS.mp4	23.74 MB
	16. Deploy Hive Application in HDFS.vtt	7.1 KB
	17. Develop Shell Wrapper to run Hive Application.mp4	35.34 MB
	17. Develop Shell Wrapper to run Hive Application.vtt	9.95 KB
	18. Overview of Scheduling and Crontab.mp4	55.8 MB
	18. Overview of Scheduling and Crontab.vtt	12.35 KB
	19. Schedule Hive Applications using Cron.mp4	113.26 MB
	19. Schedule Hive Applications using Cron.vtt	16.06 KB
	2. Getting Started with Hive.mp4	71.72 MB
	2. Getting Started with Hive.vtt	19.69 KB
	20. Recap of Application Development Life Cycle using Hive.mp4	13.36 MB
	20. Recap of Application Development Life Cycle using Hive.vtt	7.15 KB
	3. Overview of Hive Architecture.mp4	22.3 MB
	3. Overview of Hive Architecture.vtt	10.43 KB
	4. Overview of Integrating Hive Commands with Shell Scripts.mp4	43.41 MB
	4. Overview of Integrating Hive Commands with Shell Scripts.vtt	10 KB
	5. Run Hive Commands using Scripts.mp4	69.79 MB
	5. Run Hive Commands using Scripts.vtt	12 KB
	6. Override Run time Hive Configuration Properties and Variables.mp4	77.36 MB
	6. Override Run time Hive Configuration Properties and Variables.vtt	15.11 KB
	7. Getting Started Data Loader for NYSE Data using HIve.mp4	27.62 MB
	7. Getting Started Data Loader for NYSE Data using HIve.vtt	10.65 KB
	8. Design NYSE Data Loader Application.mp4	22.23 MB
	8. Design NYSE Data Loader Application.vtt	7.71 KB
	9. Create Partitioned Parquet Table for NYSE Data.mp4	46.55 MB
	9. Create Partitioned Parquet Table for NYSE Data.vtt	15.68 KB
	48. Getting Started with Spark SQL on Hadoop and Spark Cluster
	1. Getting Started with Spark SQL on Hadoop and Spark Cluster.mp4	2.84 MB
	1. Getting Started with Spark SQL on Hadoop and Spark Cluster.vtt	2.4 KB
	2. Getting Started with Data Sets and Spark SQL CLI.mp4	26.46 MB
	2. Getting Started with Data Sets and Spark SQL CLI.vtt	6.94 KB
	3. Spark SQL Metastore Architecture.mp4	25.36 MB
	3. Spark SQL Metastore Architecture.vtt	14.89 KB
	4. Overview of Spark Metastore Warehouse Directory.mp4	25.4 MB
	4. Overview of Spark Metastore Warehouse Directory.vtt	6.89 KB
	5. Launch Spark SQL CLI with Delta Lake Packages.mp4	34.44 MB
	5. Launch Spark SQL CLI with Delta Lake Packages.vtt	6.91 KB
	6. Populate Data into Delta Lake Tables using Spark SQL.mp4	84.72 MB
	6. Populate Data into Delta Lake Tables using Spark SQL.vtt	20.02 KB
	7. Run Individual Spark SQL Commands.mp4	69.58 MB
	7. Run Individual Spark SQL Commands.vtt	13.26 KB
	8. Run Spark SQL Scripts using Spark SQL CLI.mp4	75.52 MB
	8. Run Spark SQL Scripts using Spark SQL CLI.vtt	10.81 KB
	49. Build Real Time Applications using Spark SQL with Shell Wrapper
	1. Introduction to Application Development Life Cycle of Spark SQL Applications.mp4	1.91 MB
	1. Introduction to Application Development Life Cycle of Spark SQL Applications.vtt	1.84 KB
	10. Develop Shell Wrapper for Spark SQL Application.mp4	41.9 MB
	10. Develop Shell Wrapper for Spark SQL Application.vtt	9.2 KB
	2. Review the Requirements and Datasets for NYSE Data.mp4	32.59 MB
	2. Review the Requirements and Datasets for NYSE Data.vtt	7.31 KB
	3. Design NYSE Converter Application using Spark SQL and Delta.mp4	9.54 MB
	3. Design NYSE Converter Application using Spark SQL and Delta.vtt	5.5 KB
	4. Copy Files into HDFS for NYSE Converter.mp4	28.81 MB
	4. Copy Files into HDFS for NYSE Converter.vtt	6.09 KB
	5. Create External Stage Table for NYSE CSV Files.mp4	32.52 MB
	5. Create External Stage Table for NYSE CSV Files.vtt	10.66 KB
	6. Create Target Table for NYSE Data using Delta Format.mp4	54.5 MB
	6. Create Target Table for NYSE Data using Delta Format.vtt	11.07 KB
	7. Populate Data for Additional Years into Delta NYSE Table.mp4	83.11 MB
	7. Populate Data for Additional Years into Delta NYSE Table.vtt	19.27 KB
	8. Develop Spark SQL Application for NYSE Data Conversion.mp4	36.04 MB
	8. Develop Spark SQL Application for NYSE Data Conversion.vtt	6.5 KB
	9. Validate Spark SQL Application for NYSE Data Conversion.mp4	37.95 MB
	9. Validate Spark SQL Application for NYSE Data Conversion.vtt	8.82 KB
	50. Getting Started with Pyspark on Hadoop and Spark Cluster
	1. Introduction to Getting Started with Pyspark.mp4	2.08 MB
	1. Introduction to Getting Started with Pyspark.vtt	2.15 KB
	2. Launching or Getting Started Pyspark CLI.mp4	58.6 MB
	2. Launching or Getting Started Pyspark CLI.vtt	11.48 KB
	3. Overview of Spark Properties Files.mp4	30.18 MB
	3. Overview of Spark Properties Files.vtt	6.13 KB
	4. Review Data Sets to explore Pyspark APIs.mp4	30.08 MB
	4. Review Data Sets to explore Pyspark APIs.vtt	5.85 KB
	5. Getting Started with Pyspark for Data Processing.mp4	33.35 MB
	5. Getting Started with Pyspark for Data Processing.vtt	8.04 KB
	6. Read Orders and Order Items Data into Spark Data Frames.mp4	15.16 MB
	6. Read Orders and Order Items Data into Spark Data Frames.vtt	6.73 KB
	7. Process Data using pyspark Dataframe APIs.mp4	39.35 MB
	7. Process Data using pyspark Dataframe APIs.vtt	12.63 KB
	8. Overview of Spark Submit Command.mp4	34.57 MB
	8. Overview of Spark Submit Command.vtt	6.01 KB
	51. Submitting Python based Spark Applications
	1. Introduction to Submitting Python based Spark Applications.mp4	1.99 MB
	1. Introduction to Submitting Python based Spark Applications.vtt	1.64 KB
	10. Deep Dive into Spark Deploy Modes.mp4	83.5 MB
	10. Deep Dive into Spark Deploy Modes.vtt	17.32 KB
	11. Submit Spark Applications with dependencies as packages.mp4	75.97 MB
	11. Submit Spark Applications with dependencies as packages.vtt	16.08 KB
	12. Submit Spark Applications with dependencies as jars.mp4	123.6 MB
	12. Submit Spark Applications with dependencies as jars.vtt	21.11 KB
	13. Develop Shell Wrappers to submit Spark Applications.mp4	54.02 MB
	13. Develop Shell Wrappers to submit Spark Applications.vtt	10.29 KB
	2. Develop Pyspark Application for Daily Revenue.mp4	27.31 MB
	2. Develop Pyspark Application for Daily Revenue.vtt	7.54 KB
	3. Run Spark Application using spark-submit.mp4	23.13 MB
	3. Run Spark Application using spark-submit.vtt	7.92 KB
	4. Specify Paths using Environment Variables in Spark Applications.mp4	49.7 MB
	4. Specify Paths using Environment Variables in Spark Applications.vtt	8.12 KB
	5. Run Spark Application with Environment Variables in Client Mode.mp4	15.55 MB
	5. Run Spark Application with Environment Variables in Client Mode.vtt	5.4 KB
	6. Run Spark Application with Environment Variables in Cluster Mode.mp4	29.37 MB
	6. Run Spark Application with Environment Variables in Cluster Mode.vtt	6.16 KB
	7. Review Spark Application Details using Spark UI.mp4	12.49 MB
	7. Review Spark Application Details using Spark UI.vtt	4.48 KB
	8. Review YARN Logs for Spark Applications in Cluster Mode.mp4	66.69 MB
	8. Review YARN Logs for Spark Applications in Cluster Mode.vtt	9.67 KB
	9. Overview of Execution Process of Spark Applications.mp4	34.54 MB
	9. Overview of Execution Process of Spark Applications.vtt	10.15 KB
	52. Logging in Python based Spark Applications
	1. Introduction to Logging in Python baesd Spark Applications.mp4	2.92 MB
	1. Introduction to Logging in Python baesd Spark Applications.vtt	1.36 KB
	2. Run Application without logging.mp4	50 MB
	2. Run Application without logging.vtt	7.6 KB
	3. Overview of Logging Concepts such as Log Levels.mp4	28.34 MB
	3. Overview of Logging Concepts such as Log Levels.vtt	6.14 KB
	4. Getting Started with logging using Python.mp4	29.46 MB
	4. Getting Started with logging using Python.vtt	8.41 KB
	5. Changing the Log Message Format using logging.mp4	36.41 MB
	5. Changing the Log Message Format using logging.vtt	7.99 KB
	6. Add logging to Python based Spark Applications.mp4	15.09 MB
	6. Add logging to Python based Spark Applications.vtt	3.82 KB
	7. Validate Logging of Spark Application using Client Mode.mp4	24.16 MB
	7. Validate Logging of Spark Application using Client Mode.vtt	4.78 KB
	8. Validate Logging of Spark Application using Cluster Mode.mp4	54.21 MB
	8. Validate Logging of Spark Application using Cluster Mode.vtt	6.48 KB
	53. Performance Tuning of Spark Applications on Hadoop and Spark
	1. Introduction to Performance Tuning of Spark Applications on Hadoop and Spark Clu.mp4	4.56 MB
	1. Introduction to Performance Tuning of Spark Applications on Hadoop and Spark Clu.vtt	3.05 KB
	10. Determine Overall YARN Capacity.mp4	44.57 MB
	10. Determine Overall YARN Capacity.vtt	6.68 KB
	11. Overview of Spark History Server UI.mp4	20.68 MB
	11. Overview of Spark History Server UI.vtt	7.23 KB
	12. Generate Test Data for Spark Performance Tuning.mp4	46.67 MB
	12. Generate Test Data for Spark Performance Tuning.vtt	5.6 KB
	13. Develop Word Count Application using Spark.mp4	61.83 MB
	13. Develop Word Count Application using Spark.vtt	16.29 KB
	14. Develop and Validate Shell Script for Word Count.mp4	30.51 MB
	14. Develop and Validate Shell Script for Word Count.vtt	6.82 KB
	15. Overview of Jobs related to Spark Applications using Spark UI.mp4	56.48 MB
	15. Overview of Jobs related to Spark Applications using Spark UI.vtt	10.88 KB
	16. Review Environment Properties and Disabling Dynamic Allocation.mp4	50.59 MB
	16. Review Environment Properties and Disabling Dynamic Allocation.vtt	10.87 KB
	17. Overriding Spark Executor Instances to tune the performance.mp4	61.7 MB
	17. Overriding Spark Executor Instances to tune the performance.vtt	9.82 KB
	18. Determine Maximum Capacity to submit a Spark Application.mp4	22.25 MB
	18. Determine Maximum Capacity to submit a Spark Application.vtt	4.67 KB
	19. Overview of Adaptive Query Execution.mp4	45.88 MB
	19. Overview of Adaptive Query Execution.vtt	9.63 KB
	2. Delete Single Node Hadoop and Spark Cluster using Dataproc.mp4	5.05 MB
	2. Delete Single Node Hadoop and Spark Cluster using Dataproc.vtt	3.14 KB
	20. Overview of Shuffling - Part 1.mp4	23.2 MB
	20. Overview of Shuffling - Part 1.vtt	10.75 KB
	21. Overview of Shuffling - Part 2.mp4	37.87 MB
	21. Overview of Shuffling - Part 2.vtt	14.43 KB
	22. Overview of Spark Application.mp4	12.54 MB
	22. Overview of Spark Application.vtt	5.52 KB
	23. Overview of Lazy Evaluation.mp4	48.73 MB
	23. Overview of Lazy Evaluation.vtt	12.16 KB
	24. Review the code of Word Count Application.mp4	16.89 MB
	24. Review the code of Word Count Application.vtt	4.81 KB
	25. Run Spark Application with out Adaptive Query Execution.mp4	40.61 MB
	25. Run Spark Application with out Adaptive Query Execution.vtt	13.97 KB
	26. Run Spark Application using Adaptive Query Execution.mp4	41.64 MB
	26. Run Spark Application using Adaptive Query Execution.vtt	8.72 KB
	27. Overview of Spark Dynamic Allocation.mp4	60.86 MB
	27. Overview of Spark Dynamic Allocation.vtt	12.38 KB
	28. Demo on Spark Dynamic Allocation.mp4	72.7 MB
	28. Demo on Spark Dynamic Allocation.vtt	14.73 KB
	29. Running Spark Application using Dynamic Allocation.mp4	43 MB
	29. Running Spark Application using Dynamic Allocation.vtt	9.36 KB
	3. Increase GCP VM Quotas for Mutlinode Hadoop and Spark Cluster.mp4	10.12 MB
	3. Increase GCP VM Quotas for Mutlinode Hadoop and Spark Cluster.vtt	5.15 KB
	30. Overview of number of Spark Partitions.mp4	53.69 MB
	30. Overview of number of Spark Partitions.vtt	11.29 KB
	31. Delete Multinode Hadoop and Spark Cluster.mp4	9.4 MB
	31. Delete Multinode Hadoop and Spark Cluster.vtt	4.53 KB
	4. Review Quotas to setup Multinode Hadoop and Spark Cluster.mp4	12.74 MB
	4. Review Quotas to setup Multinode Hadoop and Spark Cluster.vtt	2.99 KB
	5. Setup Multinode Hadoop and Spark Cluster using GCP Dataproc.mp4	18.7 MB
	5. Setup Multinode Hadoop and Spark Cluster using GCP Dataproc.vtt	5.24 KB
	6. Setup SSH Connectivity and VS Code Workspace using Master Node.mp4	46.02 MB
	6. Setup SSH Connectivity and VS Code Workspace using Master Node.vtt	9.42 KB
	7. Review Multi Node Hadoop and Spark Clusters using Web Interfaces.mp4	16.27 MB
	7. Review Multi Node Hadoop and Spark Clusters using Web Interfaces.vtt	7.23 KB
	8. Overview of Multinode Hadoop and Spark Cluster Topology.mp4	32.38 MB
	8. Overview of Multinode Hadoop and Spark Cluster Topology.vtt	13.25 KB
	9. Computing Overall Capacity of Multinode Hadoop and Spark Clusters.mp4	14.82 MB
	9. Computing Overall Capacity of Multinode Hadoop and Spark Clusters.vtt	6.43 KB
▲ 1418 total files

Description

Обучающие видео » Компьютерные видеоуроки и обучающие интерактивные DVD » Devops

Data Engineering for Beginners: Learn SQL, Python & Spark

Год выпуска: 2/2025
Производитель: Udemy
Сайт производителя: https://www.udemy.com/course/data-engineering-essentials-sql-python-and-spark/
Автор: Durga Viswanatha Raju Gadiraju, Vinay Gadiraju
Продолжительность: 55h 56m
Тип раздаваемого материала: Видеоурок
Язык: Английский
Описание: Why Learn Data Engineering?

Data Engineering is one of the fastest-growing fields in the tech industry. Organizations of all sizes rely on Data Engineers to build and maintain the infrastructure that powers big data analytics, reporting, and machine learning. Data Engineers design, implement, and optimize data pipelines to efficiently process and manage data for business intelligence, real-time analytics, and AI applications.

With SQL, Python, and Apache Spark, Data Engineers can handle large-scale data processing efficiently. These skills are highly sought after in finance, healthcare, e-commerce, and every data-driven industry.

If you are looking for an industry-relevant and practical course that teaches you how to work with SQL, Python, Apache Spark (PySpark), and Databricks on Google Cloud Platform (GCP), this course is the perfect place to start.

What You Will Learn in This Course

This course is designed to take you from a beginner to an intermediate level in Data Engineering. You will gain hands-on experience working with SQL, Python, Apache Spark (PySpark), and Databricks by building real-world batch and streaming data pipelines.

SQL for Data Engineering (PostgreSQL)

Install and configure PostgreSQL to practice SQL queries

Learn fundamental SQL concepts such as SELECT, WHERE, JOIN, GROUP BY, HAVING, and ORDER BY

Perform advanced SQL operations including window functions, ranking, cumulative aggregations, and complex joins

Learn how to optimize SQL queries for performance and debugging

Python for Data Engineering

Understand Python fundamentals for data processing

Work with Python Collections to efficiently process structured data

Use Pandas to manipulate, clean, and analyze data

Build real-world Python projects, including a File Format Converter and a Database Loader

Learn how to troubleshoot and debug Python applications

Understand performance tuning strategies for Python-based data pipelines

Apache Spark (PySpark) for Big Data Processing

Learn Spark SQL to process structured data at scale

Work with PySpark DataFrame APIs to manipulate big data

Create and manage Delta Tables and perform CRUD operations (INSERT, UPDATE, DELETE, MERGE)

Perform advanced SQL transformations using window functions, ranking, and aggregations

Learn how to optimize PySpark jobs using Spark Catalyst Optimizer and Explain Plans

Debug, monitor, and optimize Spark jobs using Spark UI

Deploying Data Pipelines on Databricks (Google Cloud Platform - GCP)

Set up and configure Databricks on Google Cloud Platform (GCP)

Learn how to provision and manage Databricks clusters

Develop PySpark applications on Databricks and execute jobs on multi-node clusters

Understand the cost, scalability, and benefits of using Databricks for Data Engineering

Performance Tuning and Optimization in Data Engineering

Learn query performance optimization techniques in SQL and PySpark

Implement partitioning and columnar storage formats to improve efficiency

Explore debugging techniques for troubleshooting SQL and PySpark applications

Analyze Spark execution plans to improve job execution performance

Common Challenges in Learning Data Engineering and How This Course Helps

Many learners struggle with setting up a proper Data Engineering environment, finding structured learning material, and gaining hands-on experience with real-world projects.

This course eliminates these challenges by providing:

A step-by-step guide to setting up PostgreSQL, Python, and Apache Spark

Hands-on exercises that simulate real-world Data Engineering problems

Practical projects that reinforce learning and build confidence

Cloud-based Data Engineering with Databricks on Google Cloud, making it easier to work with large-scale data

Who Should Take This Course?

This course is designed for:

Beginners who want to start a career in Data Engineering

Aspiring Data Engineers who want to learn SQL, Python, Apache Spark (PySpark), and Databricks

Software Developers and Data Analysts who want to transition into Data Engineering

Data Science and Machine Learning Practitioners who need a deeper understanding of data pipelines

Anyone interested in Big Data, ETL processes, and cloud-based Data Engineering

Why Take This Course?

Beginner-Friendly Approach

This course starts with the fundamentals and gradually builds up to advanced topics, making it accessible for beginners.

Hands-On Learning with Real-World Projects

You will work on real-world projects to reinforce your skills and gain practical experience in building Data Pipelines.

Cloud-Based Training on Databricks (GCP)

This course teaches cloud-based Data Engineering using Databricks on Google Cloud, a platform widely used by companies for Big Data processing and machine learning.

Comprehensive Curriculum Covering All Key Data Engineering Skills

This course covers SQL, Python, Apache Spark (PySpark), Databricks, ETL, Big Data Processing, and Performance Optimization—all essential skills for a Data Engineer.

Performance Tuning and Debugging

You will learn how to analyze Spark execution plans, optimize SQL queries, and debug PySpark jobs, which are crucial for real-world Data Engineering projects.

Lifetime Access and Updates

You get lifetime access to the course content, which is regularly updated to keep up with industry trends and new technologies.

Course Features

Step-by-step instructions with detailed explanations

Hands-on exercises to reinforce learning

Real-world projects covering batch and streaming data pipelines

Complete Databricks setup guide for Google Cloud

Performance optimization techniques for SQL and PySpark

Best practices for debugging and tuning Spark jobs

Enroll Today and Start Your Data Engineering Journey

If you are serious about learning Data Engineering and want to master SQL, Python, Apache Spark (PySpark), and Databricks on Google Cloud, this course will provide you with the essential skills and hands-on experience needed to succeed in this field.

Take the first step in your Data Engineering journey today—enroll now!
[spoiler="Содержание"]
01. Introduction to Data Engineering Essentials using SQL, Python, and PySpark
1. Introduction to Data Engineering Essentials Course (02:41)
2. Overview of our support to Data Engineering Essentials course (04:40)
3. Overview of SQL topics covered in the course (06:57)
4. Overview of Python topics covered in the course (07:02)
5. Overview of Getting Started with GCP related to the course (04:09)
6. Overview of Spark and Databricks Environment related topics (04:46)
7. Detailed outline of Spark SQL Topics in the course (06:35)
8. Detailed outline of Pyspark Topics in the course (05:10)
9. Detailed outline of ELT Data Pipelines on Databricks (02:27)
10. Overview of Performance Tuning of Spark covered in the course (05:14)

02. Getting Started with SQL for Data Engineering
1. Introduction to SQL for Data Engineering (01:36)
2. Overview of Application Architecture and RDBMS (03:58)
3. Overview of Database Technologies and relevance of SQL (06:16)
4. Overview of Purpose Built Databases (06:00)
5. Overview of Data Warehouse and Data Lake (03:13)
6. Usage of RDBMS and Data Warehouse technologies (03:36)
7. Differences and Similarities between RDBMS and Data Warehouse Technologies (03:52)

03. Setup Tools for Data Engineering Essentials
1. Introduction to Setting up Tools for Data Engineering Essentials (02:29)
2. Setup VS Code on Windows (03:22)
3. Setup Python 3.9 on Windows (05:42)
4. Configure Environment Variable PATH for Python on Windows (04:18)
5. Overview of learning Python using Python CLI (03:22)
6. Integrate VSCode with Python on Windows (05:02)
7. Install Postgres 14 on Windows 11 (05:13)
8. Getting Started with pgAdmin on Windows (02:36)
9. Getting Started with pgAdmin on Mac (05:31)
10. Conclusion of Setting up Tools for Data Engineering Essentials (01:45)

04. Setup Application Tables and Data in Postgres Database
1. Overview of Postgres Database Server and pgAdmin (08:07)
2. Overview of Database Connection Details (07:03)
3. Overview of Connecting to External Databases using pgAdmin (04:19)
4. Create Application Database and User in Postgres Database Server (03:47)
5. Clone Data Sets from Git Repository for Database Scripts (02:50)
6. Register Server in pgAdmin using Application Database and User (03:58)
7. Setup Application Tables and Data in Postgres Database (07:19)
8. Overview of pgAdmin to write SQL Queries (11:41)

05. Writing Basic SQL Queries
1. Review Data Model Diagram (04:33)
2. Define Problem Statement for SQL Queries (01:42)
3. Filtering Data using SQL Queries (06:27)
4. Total Aggregations using SQL Queries (05:26)
5. Group By Aggregations using SQL Queries (07:29)
6. Order of Execution of SQL Queries (07:31)
7. Rules and Restrictions to Group and Filter Data in SQL queries (05:19)
8. Filter Data based on Aggregated Results using Group By and Having (04:31)
9. Inner Joins using SQL Queries (05:15)
10. Outer Joins using SQL Queries (07:12)
11. Filter and Aggregate on Join Results using SQL (03:57)
12. Overview of Database Views (06:02)
13. Overview of Common Table Expressions or CTEs (04:03)
14. Outer Join with Additional Conditions in SQL Queries (08:10)
15. Explanation about Fix of SQL Queries with Filtering on Outer Join Results (04:20)

06. Cumulative Aggregations and Ranking in SQL Queries
1. Introduction to Cumulative Aggregations and Ranking in SQL Queries (01:29)
2. Overview of CTAS to create tables based on Query Results (04:26)
3. Create Tables for Cumulative Aggregations and Ranking (02:11)
4. Overview of OVER and PARTITION BY Clause in SQL Queries (06:59)
5. Compute Total Aggregation using OVER and PARTITION BY in SQL Queries (01:47)
6. Overview of Ranking in SQL (03:09)
7. Compute Global Ranks using SQL (03:54)
8. Compute Ranks based on key using SQL (04:15)
9. Rules and Restrictions to Filter Data based on Ranks in SQL (02:45)
10. Filtering based on Global Ranks using Nested Queries and CTEs in SQL (05:04)
11. Filtering based on Ranks per Partition using Nested Queries and CTEs in SQL (04:51)
12. Create Students table with Data for ranking using SQL (03:11)
13. Difference between rank and dense rank using SQL (05:39)

07. SQL Troubleshooting and Debugging Guide
1. Introduction to SQL Troubleshooting and Debugging Guide (01:48)
2. Overview of Database Connectivity Issues (07:17)
3. Validate and Setup Telnet on Mac or PC (04:12)
4. Validate Connectivity to Database Server using telnet (04:27)
5. Troubleshoot Database Connectivity Issue with Correct Host Details (06:12)
6. Current Databases and Users in Postgres Database Server (04:38)
7. Troubleshoot Database Credentials and Permissions Issues (07:55)
8. Overview of Compilation of SQL Queries (05:31)
9. Troubleshooting Syntax Errors in SQL Queries (04:03)
10. Troubleshooting Semantec Errors in SQL Queries (04:56)
11. Overview of Bugs in SQL Queries (02:56)
12. Development Best Practices with tips to troubleshoot SQL bugs (03:47)
13. Develop Initial Solution based on the requirement (04:14)
14. Identify and Troubleshoot Bugs in SQL Queries (08:06)
15. Develop Solution using Development Best Practices (04:39)

08. Performance Tuning of SQL Queries
1. Introduction to Performance Tuning of SQL Queries (03:32)
2. Overview of SQL Compilation Process and Explain Plans (04:26)
3. Generate Explain Plans for SQL Queries (05:59)
4. Review Tables used for Performance Tuning of SQL Queries (07:03)
5. Review Data Storage Internals for Tables and Indexes (05:15)
6. Review key terms used in Explain Plans for SQL Queries (03:42)
7. Interpret Explain Plans for Basic SQL Queries (05:23)
8. Review the Common Application Scenarios for Performance Tuning (03:01)
9. Write SQL Queries for Customer Orders (05:37)
10. Performance Testing of SQL Queries using Stored Procedure (04:05)
11. Add Required Indexes to tune performance of SQL Queries (07:26)
12. Guidelines on adding Indexes on Tables for SQL Queries (03:40)
13. Interpreting the explain plan for SQL Queries using Indexes (05:38)
14. Conclusion of Performance Tuning of SQL Queries (02:42)

09. Exercises for Basic SQL Queries
1. Simple Exercises for Filtering and Aggregations (04:23)
2. Exercises on Joins and Aggregations using SQL (02:50)

10. Solutions for Basic SQL Queries
1. Solutions for Filtering and Aggregations (04:56)
2. Solutions for Filtering and Aggregations (05:31)
3. Validate Data and Review Data Model Diagram (03:25)
4. Solution for Exercise 1 to get Customer Order Count (07:50)
5. Solution for Exercise 2 to get Dormant Customers using Outer Join (10:55)
6. Solution for Exercise 3 to get Revenue Per Customer using Outer Join (07:49)
7. Solution for Exercise 4 to get Revenue Per Category (07:37)
8. Solution for Exercise 5 to get Product Count Per Department (08:29)

11. Getting Started with Python
1. Setup Visual Studio Workspace for Python Application Development (06:04)
2. Setup Notebook Environment in VS Code Workspace (06:48)
3. Overview of VS Code Notebook Environment (04:39)
4. Overview of Cells in VS Code Notebook (06:06)
5. Defining Functions in VS Code Notebooks (05:49)
6. Run the Code in VS Code Notebook Cell by Line (05:18)
7. Constants and Variables in Python (05:43)
8. Overview of Python Data Types (04:11)
9. Getting help on Python Variables and Functions (06:15)
10. Pre-Defined String Manipulation Functions (06:37)
11. Overview of Python Lists (06:42)
12. Loops and Conditions in Python (08:49)
13. User Defined Functions in Python (06:29)

12. Python Collections for Data Engineering
1. Overview of File IO using Python (08:22)
2. Read Data from CSV File into Python List (03:55)
3. Overview of Python Collections (08:26)
4. Getting Started with Processing Python Lists (03:17)
5. Overview of Lambda Functions in Python (06:30)
6. Usage of Lambda Functions (03:31)
7. Filter Data in Python Lists using filter and lambda (05:43)
8. Get unique values from list using map and set (05:23)
9. Sort Python lists using key (07:42)
10. Overview of JSON Strings and Files (03:54)
11. Read JSON Strings to Python dicts or lists (06:34)
12. Read JSON Schemas from file to Python dicts (06:18)
13. Overview of Processing JSON Data using Python (03:04)
14. Extract Details from Complex JSON Arrays using Python (03:39)
15. Sort Data in JSON Arrays using Python (04:17)
16. Create Function to get Column Details from Schemas JSON File (05:28)

13. Data Processing using Pandas Dataframe APIs
1. Overview of Pandas for Data Processing (04:43)
2. Overview of Reading CSV Data using Pandas (04:38)
3. Read Data from CSV Files to Pandas Dataframes (04:43)
4. Filter Data in Pandas Dataframe using query (07:06)
5. Get Count by Status using Pandas Dataframe APIs (05:10)
6. Get count by Month and Status using Pandas Dataframe APIs (04:32)
7. Create Dataframes using dynamic column list on CSV Data (04:47)
8. Performing Inner Join between Pandas Dataframes (04:21)
9. Perform Aggregations on Join results (05:00)
10. Sort Data in Pandas Dataframes (06:45)
11. Overview of Writing Pandas Dataframes to Files (03:10)
12. Write Pandas Dataframes to JSON Files (07:16)

14. Project 1 - File Format Converter using Python
2. Get File Names to be processed using glob (05:53)
3. Get Column Names using Schemas File (04:03)
4. Get Data Set Names from File Names or Paths using regular expressions (05:34)
5. Read CSV Data into Pandas Dataframe with Schema Dynamically (05:04)
6. Generate File Paths for Target JSON Files Dynamically (04:01)
7. Recap of Writing Pandas Dataframe to JSON File (02:24)
8. Write Pandas Dataframe to JSON Files (04:15)
9. Modularize File Format Converter for Dataset (05:18)
10. Wrapper to Process all Data Sets (06:37)
11. Setup Project for File Format Converter using Python (04:17)
12. Install Dependencies for the Python Project using pip (03:30)
13. Add Core Logic to Python Application (04:10)
14. Overview of Run-time Arguments and Environment Variables (02:03)
15. Using Run Time Arguments in Python Applications (05:32)
16. Overview of Environment Variables (04:21)
17. Setting Environment Variables on Windows or Mac or Linux (04:53)
18. Use Environment Variables in Python Applications (05:34)
19. Use Environment Variables in File Format Converter (09:15)
20. Pass JSON Array as argument to Python Applications (05:51)
21. Pass Data Sets as Run Time Arguments to File Format Converter (09:20)
22. Exception Handling in Python Applications (03:35)
23. Raising Exceptions in Python Applications (04:07)
24. Exception Handling in File Format Converter Application (04:38)

15. Project 2 - Files to Database Loader
2. Install Python Dependencies for Pandas and Database Integration (02:11)
3. Run Queries from Notebook using SQL Magic (07:58)
4. Validate Pandas and SQL Integration (05:21)
5. Write CSV Data from File to Database Table (05:43)
6. Write CSV Data from Files to Database Tables in Chunks (07:05)
7. Overview of Deploying File to DB Loader Project (09:12)

16. Troubleshooting and Debugging Python Issues
1. Introduction to Troubleshooting and Debugging Python issues (02:48)
2. Guidelines for Troubleshooting and Debugging Python related Issues (01:47)
3. Overview of Database Connectivity using Python Applications (07:32)
4. Overview of Database Connectivity using Python (04:15)
5. Troubleshoot Network Connectivity to the Database Server using telnet (03:44)
6. Troubleshoot Module Related issues for Database Connectivity using Python (05:08)
7. Troubleshoot Credentials Related issues for Database Connectivity using Python (03:08)
8. Overview of Python process to run Python Applications (02:40)
9. Troubleshooting Compilation Errors in Python (04:55)
10. Troubleshooting Run Time Errors in Python (03:58)
11. Overview of Software Development Life Cycle (03:31)
12. Overview of Unit Testing or Validation of Applications (03:11)
13. Overview of Debugging VS Code Notebooks using Debug Feature (09:14)
14. Debug VS Code Notebooks using Debug Feature (09:46)
15. Getting Started with Debugging of Python Programs using VS Code (02:27)
16. Recap of running File Format Converter application (06:26)
17. Debug Python Application using VS Code with breakpoints (06:46)
18. Managing Breakpoints for Debugging in VS Code (02:56)
19. Conclusion to Troubleshooting and Debugging Python Issues (02:02)

17. Performance Tuning of Python Applications
1. Introduction to Performance of Python Applications (07:00)
2. Setup Database Loader Python Application (05:42)
3. Ensure Postgres Database is setup for file to db loader Python Application (09:22)
4. Cleanup the tables to run file to db loader application (03:08)
5. Run and Validate File to DB Loader Application (06:07)
6. Fix the error message in file to db loader application (02:29)
7. Overview of Execution of file to db loader application (07:16)
8. Performance Tuning using Chunksize in Pandas (06:50)
9. Review Pandas Data Frame API to load data into the target table (08:30)
10. Overview of multi or batch insert into Database Tables (04:53)
11. Develop application for multiprocessing (03:06)
12. Getting Started with Multiprocessing using Python (05:14)
13. Invoking User Defined Functions using multiprocessing in Python (06:21)
14. Refactor File to Database Loader Application for Multiprocessing (04:38)
15. Add Parallel Processing to file to db loader Python Application (04:41)
16. Validate File to DB Loader Application with Multiprocessing (07:07)
17. Understanding the concept of Multiprocessing in Python (08:01)
18. Performance Tuning Scenarios of Python Applications (05:11)

18. Getting Started with GCP
1. Introduction to Getting Started with GCP (01:00)
2. Pre-requisite Skills to Sign up for course on GCP Data Analytics (02:03)
3. Overview of Cloud Platforms (04:00)
4. Overview of Google Cloud Platform or GCP (03:20)
5. Overview of Signing for GCP Account (01:42)
6. Create New Google Account using Non Gmail Id (02:17)
7. Sign up for GCP using Google Account (03:16)
8. Overview of GCP Credits (03:36)
9. Overview of GCP Project and Billing (02:11)
10. Overview of Google Cloud Shell (03:28)
11. Install Google Cloud SDK on Windows (04:38)
12. Initialize gcloud CLI using GCP Project (03:25)
13. Reinitialize Google Cloud Shell with Project id (03:04)
14. Overview of Analytics Services on GCP (02:20)
15. Conclusion to Get Started with GCP for Data Engineering (01:05)

19. Overview of Big Data and Data Lakes
1. Different Types of Databases (01:59)
2. Usecases for Different Types of Databases (04:01)
3. Technologies for Different Types of Databases (03:41)
4. Volumes for Different Types of Databases (03:29)
5. Overview of Big Data (04:19)
6. Evolution of Big Data Technologies (02:33)
7. Overview of Data Lake using Hadoop eco system (06:14)
8. Limitations of Hadoop eco system (05:31)
9. Overview of Modern Data Lakes on Cloud (02:36)
10. Implementation of Modern Data Lakes on Cloud (03:23)
11. Advantages of Modern Data Lakes on Cloud (04:11)

20. Overview of Spark and Spark Architecture
1. Overview of Data Processing (05:42)
2. Overview of Data Processing Libraries (01:56)
3. Setup Environment to explore Pandas, Dask and Pyspark (04:19)
4. Code Examples of Pandas, Dask and Pyspark (06:21)
5. Differences between Pandas, Dask and Pyspark (06:13)
6. Overview of Distributed Computing (05:16)
7. Overview of Official Documentation of Apache Spark (03:37)
8. Overview of Spark Key Features and Platforms (05:01)
9. Overview of Spark Infrastructure (03:58)
10. Overview of Spark Cluster using Databricks (06:14)
11. Overview of Executors in Spark Cluster (03:03)
12. Overview of Spark Glossary (01:44)
13. Understand Spark Key Terms (06:42)

21. Setup Databricks Environment using GCP
1. Overview of Databicks on GCP (01:08)
2. Signing up for Databricks on GCP (04:21)
3. Create Databricks Workspace on GCP (04:46)
4. Getting Started with Databricks Clusters on GCP (03:17)
5. Getting Started with Databricks Notebook (03:51)
6. Overview of Databricks on GCP (07:11)
7. High level architecture of Databricks (05:19)
8. Setup Databricks CLI on Mac or Windows (05:07)
9. Overview of Databricks CLI and other clients (05:20)
10. Configure Databricks CLI on Mac or Windows (05:18)
11. Troubleshoot issues to configure Databricks CLI (05:11)
12. Overview of Databricks CLI Commands (05:20)
13. Setup Data Repository for Data Sets (01:59)
14. Setup Data Sets in DBFS using Databricks CLI Commands (04:50)

22. Basic Transformations using Spark SQL
1. Process Data in DBFS using Databricks Spark SQL (05:03)
2. Getting Started with Spark SQL Example using Databricks (04:44)
3. Create Temporary Views using Spark SQL (06:34)
4. Exercise to create temporary views using Spark SQL (01:27)
5. Spark SQL Query to compute Daily Product Revenue (06:10)
6. Save Query Result to DBFS using Spark SQL (04:25)
7. Overview of Pyspark Examples on Databricks (01:04)
8. Process Schema Details in JSON using Pyspark (07:32)
9. Create Dataframe with Schema from JSON File using Pyspark (06:03)
10. Transform Data using Spark APIs (04:13)
11. Get Schema Details for all Data Sets using Pyspark (04:08)
12. Convert CSV to Parquet with Schema using Pyspark (05:01)

23. Create Delta Tables using Spark SQL
1. Introduction to Creating Delta Tables using Spark SQL (01:34)
2. Overview of Supported Providers for Spark Metastore Tables (02:07)
3. Create Database and Review the Details (05:58)
4. Create and Review Managed Spark Metastore Table using Delta Format (04:31)
5. Copy Data into Spark Metastore Managed Table (04:41)
6. Validate Data in Spark Metastore Managed Table (03:17)
7. Create and Review External Spark Metastore Table using Delta Format (04:08)
8. Insert Data into Spark Metastore External Table (03:13)
9. Validate Data in Spark Metastore External Table (03:00)
10. Overview of Spark Metastore (04:36)
11. Difference Between Managed and External Spark Metastore Tables (06:38)
12. Perform CRUD Operations on Delta Tables in Spark Metastore (06:14)
13. Using Merge to Update and Insert into Delta Tables in Spark Metastore (09:31)
14. Conclusion of Creating Delta Tables using Spark SQL (02:15)

24. Pre-Defined Functions in Spark SQL
1. Overview of Functions in Spark SQL (03:08)
2. Validate Functions in Spark SQL (04:01)
3. Overview of String Manipulation Functions in Spark SQL (01:54)
4. Case Conversion and Length of Strings using Spark SQL (03:51)
5. Extract Substring using substr in Spark SQL (05:19)
6. Extract Substrings from Delimited Strings using split in Spark SQL (04:33)
7. Trimming Characters or Strings using Spark SQL (09:32)
8. Padding Characters to Strings using Spark SQL (07:32)
9. Reverse and Concatenate Strings using Spark SQL (07:06)
10. Overview of Date Manipulation Functions in Spark SQL (01:37)
11. Overview of Standard Date and Timestamp in Spark SQL (03:27)
12. Date Arithmetic using Spark SQL Functions (04:40)
13. Overview of trunc and date_trunc in Spark SQL (05:36)
14. Extract Information from Date or Time using Spark SQL (08:23)
15. Convert Non Standard Dates or Timestamps to Standard Ones using Spark SQL (07:39)
16. Extract Information using Calendar Functions from Date or Timestamp using Spark (02:51)
17. Dealing with Unix Timestamp using Spark SQL (08:54)
18. Overview of Numeric Functions in Spark SQL (12:15)
19. Data Type Conversion using Spark SQL (06:15)
20. Overview of Handling Null Values using Spark SQL (03:51)
21. Replace Null Values with default values using nvl and coalesce in Spark SQL (07:15)
22. Conditional Logic on Null Values using nvl2 and case in Spark SQL (06:08)
23. Overview of Case and When in Spark SQL (03:07)
24. Using CASE and WHEN for conditional logic in Spark SQL (06:09)
25. Aggregate using CASE and WHEN in GROUP BY in Spark SQL (04:00)
26. Word Count Query using Pre-defined Functions in Spark SQL (06:34)
27. Exercises for Pre-defined functions in Spark SQL (02:02)
28. Solutions for Exercises 1 and 2 on Pre-defined Functions in Spark SQL (08:27)
29. Solutions for Exercises 3 and 4 on Pre-defined Functions in Spark SQL (05:49)
30. Solutions for Exercises 5 and 6 on Pre-defined Functions in Spark SQL (09:59)
31. Solutions for Exercises 7 and 8 on Pre-defined Functions in Spark SQL (10:12)

25. Setup Spark Metastore Tables for Basic Transformations
1. Introduction to Basic Transformations using Spark SQL (01:45)
2. Prepare Spark Metastore Tables for Basic Transformations (03:47)
3. Projecting Data using Spark SQL (06:01)

26. Filtering Data using Spark SQL Queries
1. Filtering Data using Equal Condition in Spark SQL (04:56)
2. Using IN, LIKE and BETWEEN in Spark SQL Queries (04:48)
3. Filter Data using Boolean AND in Spark SQL Queries (04:25)
4. Filter Data using Boolean OR in Spark SQL Queries (04:19)
5. Dealing with NULLS while Filtering Data in Spark SQL Queries (03:15)

27. Aggregations using Spark SQL Queries
1. Perform Total Aggregations using Spark SQL Queries (06:45)
2. Overview of Aggregations using GROUP BY in Spark SQL Queries (03:59)
3. GROUP BY Examples using Spark SQL Queries (04:45)
4. Order of Execution of Spark SQL Queries (06:07)
5. Filter Data based on Aggregate Results using HAVING in Spark SQL Queries (06:23)

28. Joins using Spark SQL Queries
1. Overview of Joins in Spark SQL Queries (04:20)
2. Inner Join using Spark SQL Queries (05:47)
3. Concepts Behind Inner Joins in Spark SQL (05:03)
4. Outer Joins using Spark SQL Queries (08:39)
5. Example - Inner Join along with GROUP BY using Spark SQL Queries (03:36)
6. Example - Outer Join along with GROUP BY using Spark SQL Queries (03:29)
7. Example - Filtering and Outer Joins along with GROUP BY in Spark SQL Queries (03:33)
8. Example - Filtering and Outer Joins along with GROUP BY in Spark SQL Queries (03:58)

29. Sorting using Spark SQL Queries
1. Sorting Data using Spark SQL Queries (04:01)
2. Dealing with Nulls while Sorting Data using Spark SQL Queries (03:28)

30. Copy Query Results into Spark Metastore Tables
1. Overview of Copying Query Results into Spark Metastore Tables (02:09)
2. Query to Compute Daily Revenue using Spark SQL (02:16)
3. Copy Query Results into Spark Metastore Tables using CTAS (04:25)
4. Copy Query Results into Spark Metastore Tables using INSERT (03:38)
5. Design Pipeline using CTAS and INSERT in Spark SQL (09:02)
6. Copy Query Results into Spark Meatstore Tables using MERGE (07:14)

31. Ranking using Spark SQL Windowing Functions
1. Ranking using Spark SQL Windowing Functions (01:31)
2. Create Temporary View for ranking using Spark SQL Windowing Functions (01:36)
3. Compute Global Rank using Spark SQL Windowing Functions (05:37)
4. Compute Ranks Per Key using Spark SQL Windowing Functions (03:55)
5. Difference Between rank and dense_rank (04:10)
6. Filter on Ranks using Spark SQL Windowing Functions (06:59)

32. Processing JSON like Data using Spark SQL
1. Overview of JSON (04:41)
2. Creating Spark Metastore Tables with Array Type Columns (04:01)
3. Dealing with Array Type Columns using Spark SQL Queries (04:42)
4. Creating Spark Metastore Tables with Struct Type Columns (03:03)
5. Projecting Data From Struct Type Fields in Spark SQL (03:54)
6. Creating Spark Metastore Tables with Array of Struct Column (03:48)
7. Dealing with Array of Struct Type Columns using Spark SQL Queries (04:30)
8. Overview of Important Functions to Process JSON Data in Spark SQL (03:32)
9. Generate Array Type Columns from Regular Columns in Spark SQL (04:35)
10. Generate Array of Struct Type Columns from Regular Columns in Spark SQL (03:53)
11. Processing Delimited Strings using Spark SQL Queries (04:08)

33. Getting Started with Pyspark Data Frame APIs
1. Overview of Pyspark Examples on Databricks (01:04)
2. Process Schema Details in JSON using Pyspark (07:32)
3. Create Dataframe with Schema from JSON File using Pyspark (06:03)
4. Transform Data using Spark APIs (04:13)
5. Get Schema Details for all Data Sets using Pyspark (04:08)
6. Convert CSV to Parquet with Schema using Pyspark (05:01)

34. Create Spark Data Frames using Pyspark Data Frame APIs
1. Create Spark Data Frame using Pyspark Data Frame APIs (04:26)
2. Introduction to Processing JSON like Data using Spark SQL (01:03)
3. Overview of Data Processing using Conventional loops (06:50)
4. Overview of Data Frame Concepts (05:43)
5. Advantages of Pyspark Data Frames (03:40)
6. Overview of Spark Data Frames and their Characteristics (05:13)
7. Projecting Data in Spark Data Frames using Select (03:57)
8. Using drop to drop columns from Spark Data Frame (02:28)
9. Applying functions on Spark Data Frame Columns (05:55)
10. Using withColumn to apply transformations on Spark Data Frames (03:19)
11. Overview of Writing Data in Data Frame to Delta Files (04:49)

35. Basic Transformations using Pyspark Data Frame APIs
1. Overview of Basic Transformations using Pyspark Data Frame APIs (03:00)
2. Overview of Row Level Transformations (05:15)
3. Apply Row Level transformations using Pyspark Data Frame APIs (07:27)
4. Filtering Data using Pyspark Data Frame APIs (04:39)
5. Filtering Data with Multiple Conditions using Pyspark Data Frame APIs (05:54)
6. Perform Aggregations by Key using Spark Data Frame APIs (04:31)
7. Perform Aggregations by Key using Spark Data Frame APIs (06:03)
8. Perform Aggregations by Key using Spark Data Frame APIs (03:57)
9. Sorting Data using Spark Data Frame APIs (05:20)
10. Composite Sorting using Spark Data Frame APIs (05:56)
11. Develop Spark SQL Queries for Sorting Data (02:50)
12. Review Data Set with Nulls for Sorting using Spark Data Frame APIs (05:26)
13. Dealing with Nulls while Sorting the Data in Spark Data Frames (04:09)

36. Joining Data using Spark Data Frame APIs
1. Introduction to Joining Data using Spark Data Frame APIs (01:41)
2. Create Data Frames to Join using Spark Data Frame APIs (05:04)
3. Review Syntax for join using Spark Data Frame APIs (02:44)
4. Inner Join using Spark Data Frame APIs (05:31)
5. Join and other Spark Data Frame APIs to process the data (06:57)
6. Analyze Data for outer joins using Spark Data Frame APIs (03:26)
7. Left Outer Join using Spark Data Frame APIs (06:08)
8. Right Outer Join using Spark Data Frame APIs (04:19)
9. Equivalent Spark SQL Queries for Joins (05:05)

37. Ranking using Pyspark Data Frame APIs
1. Introduction to Ranking using Spark Data Frame APIs (05:52)
2. Syntax for ranking using Spark Data Frame APIs (02:59)
3. Compute Global Ranks using Spark Data Frame APIs (05:02)
4. Filter Based on Global Ranks using Spark Data Frame APIs (04:19)
5. Compute Ranks per Partition using Spark Data Frame APIs (03:30)
6. Filter Based on Ranks Per Partition using Spark Data Frame APIs (03:46)
7. Difference Between rank and dense_rank (07:27)

38. Integration of Spark SQL and Pyspark Data Frame APIs
1. Introduction to Integration of Spark SQL and Pyspark Data Frame APIs (01:42)
2. Run Spark SQL Queries on Spark Data Frames (05:19)
3. Create Spark Metastore Tables using Data Frames (05:00)
4. Insert into Spark Metastore Tables using Data Frames (04:03)
5. Read Data from Spark Metastore Table to Data Frames (03:33)
6. Process Data in Spark Metastore Tables using Data Frame APIs (05:37)
7. Manage Spark Metastore Database Objects using Spark APIs (05:07)

39. ELT Data Pipelines using Databricks
1. Overview of Databricks Workflows (03:10)
2. Pass Arguments to Databricks Python Notebooks (07:36)
3. Pass Arguments to Databricks SQL Notebooks (03:16)
4. Create and Run First Databricks Job (07:31)
5. Run Databricks Jobs and Tasks with Parameters (05:40)
6. Create and Run Orchestrated Pipeline using Databricks Job (06:54)
7. Import ELT Data Pipeline Applications into Databricks Environment (02:56)
8. Spark SQL Application to Cleanup Database and Datasets (03:52)
9. Review File Format Converter Pyspark Code (05:11)
10. Review Databricks SQL Notebooks for Tables and Final Results (03:57)
11. Validate Applications for ELT Pipeline using Databricks (07:36)
12. Build ELT Pipeline using Databricks Job in Workflows (09:22)
13. Run and Review Execution details of ELT Data Pipeline using Databricks Job (05:00)

40. Performance Tuning of Spark - Catalyst Optimizer
1. Getting Started with Performance Tuning using Spark on Databricks (03:12)
2. Overview of Spark Catalyst Optimizer (04:16)
3. Review Explain Plan for Spark Dataframe logic using Spark UI (03:41)
4. Review Explain Plan for Spark SQL logic using Spark UI (03:04)
5. Generate Explain Plans on Spark Dataframes using explain function (03:48)
6. Generate Explain Plans on Spark SQL Queries using explain command (02:33)
7. Interpreting Explain Plan for Spark SQL Query (04:57)
8. Overview of Spark Architecture (03:27)
9. Understand Filter and Broadcast of Orders Data (05:31)
10. Understand Join and Aggregation for Daily Product Revenue (04:18)

41. Performance Tuning of Spark - Cluster Configuration
1. Introduction to Databricks Cluster Configuration (01:59)
2. Difference Between All Purpose and Jobs Clusters (05:17)
3. Setting up All Purpose Databricks Compute Clusters (04:46)
4. Understand the size of the data using dbutils (06:03)
5. Create Multinode Databricks Cluster with Auto Scaling (02:58)
6. Overview of Auto Scaling of Databricks Clusters (03:21)
7. Performance Tuning of Cluster using Auto Scaling (04:40)
8. Analyze Airlines Data using Spark SQL (02:58)
9. Setup Databricks Job Compute Clusters using Workflows (04:27)
10. Review Running Job Details using Spark UI (03:03)
11. Review Completed Job Details using Spark UI (03:20)

42. Performance Tuning while inferring schema from CSV or JSON files
1. Overview of Inferring Schema using CSV or JSON Files (01:45)
2. Steps to convert CSV or JSON Files to Parquet or Delta Files (03:50)
3. Overview of CSV or JSON Files (05:08)
4. Overview of overhead for inferring schema (05:47)
5. Performance Tuning to infer schema of Spark Dataframe (04:58)

43. Performance Tuning using Columnar File Format and Partitioning Strategy
1. Introduction to Performance Tuning while storing data in Data Lake (01:15)
2. Side effects of using CSV Files in Data Lake (04:28)
3. Review the side effects of using CSV Files in Data Lake (03:02)
4. Restructure CSV Data to Columnar Format using Pyspark (04:55)
5. Compute Size of restrucutured data using Parquet File Format (03:53)
6. Run Operations on Partitioned Parquet Data (04:21)
7. Overview of Performance Assessment of Spark Jobs (03:44)
8. Review Performance Details of Spark Operations on Parquet Files (07:54)
9. Overview of Columnar File Formats in Spark and Databricks (06:49)
10. Overview of Folder Structure for Partitioned Data in Spark (04:48)
11. Evaluate Requirements against Partition Pruning (06:32)
12. Solutions on Airlines Data for Performance Tuning using Partitioning (07:00)
13. Review Execution Details for Performance Tuning using Partitioning (10:17)
14. Get Airlines Data using Date Range without Partition Pruning (06:23)
15. Parameterize Spark SQL Solution for Partition Pruning (03:43)
16. Add Condition for Partition Pruning (04:41)
17. Redesign Partition Strategy to tune the performance (05:21)
18. Recap of Spark Performance Tuning Scenarios (04:52)

44. Setup Hadoop and Spark Cluster using Dataproc
1. Introduction to Setup Hadoop and Spark Cluster using Dataproc (03:18)
2. Overview of different Spark Platforms on Cloud (06:42)
3. Overview of Hadoop and Spark Cluster Types and Architecture (10:46)
4. Setup Single Node Hadoop and Spark Cluster using Dataproc (03:42)
5. Validate SSH Connectivity to the Dataproc Cluster (06:28)
6. Convert IP Address to Static for Dataproc Cluster (03:58)
7. Setup Project using VS Code Remote Development (15:46)
8. Setup Local Data Sets on Hadoop and Spark Cluster (02:18)
9. Getting Started with HDFS Commands to Manage Files (07:30)
10. Getting Started with Spark CLI using Python (04:52)
11. Getting Started with Spark CLI using Scala (03:59)
12. Getting Started with Spark CLI using SQL (05:00)
13. Stopping the Cluster and Understanding the costs (05:36)
14. Download VS Code Workspace and Delete Cluster (04:17)

45. Recap of important Linux Commands for Data Engineering
1. Introduction to Linux Commands and Scripts for Data Engineers (02:56)
2. Overview of SSH to connect to remote Servers (21:33)
3. Overview of Profile in Linux Shell (07:16)
4. Overview of Environment Variables in Linux (06:19)
5. Understanding PATH Environment Variable (18:13)
6. Creating Folders in Linux using mkdir (08:54)
7. Copy Files and Folders in Linux using cp command (09:39)
8. Move Files and Folders in Linux using mv command (06:31)
9. Delete Files and Folders in Linux using rm command (08:37)
10. Listing Files and Folders using ls command (10:11)
11. Searching for files using find command in Linux (06:15)
12. Standard Directories in Linux (08:44)
13. Troubleshooting issues in Linux using grep command (13:46)
14. Overview of Shell Scripts (10:40)
15. Running and Debugging Shell Scripts with Arguments (09:26)
16. Overview of Hadoop and Spark Executables (07:44)

46. Mastering Hadoop HDFS Commands and Concepts
1. Introduction to Mastering Hadoop HDFS Commands and Concepts (02:20)
2. Start the Hadoop and Spark Cluster using Dataproc (02:22)
3. Create Hadoop and Spark Cluster and Setup VS Code Workspace (07:12)
4. Overview of important HDFS Commands (02:51)
5. Create Folder and Copy Files into HDFS using Commands (08:19)
6. Programmatically Copy files into HDFS using Python (11:22)
7. Overview of Multinode Hadoop Cluster (12:19)
8. Review Important Properties of HDFS (06:20)
9. Review HDFS Properties on Dataproc Cluster using VS Code (04:05)
10. Overview of local storage of files (09:35)
11. Setup Data Sets to understand HDFS Concepts (05:51)
12. Overview of Distributed Storage of files in HDFS (06:39)
13. Determining Number of Blocks for each file (09:31)
14. Overview of Blocks related to files in HDFS (09:11)
15. Overview of Replication related to files in HDFS (08:22)
16. Physical Storage of HDFS File Blocks (06:53)
17. Overview of HDFS Namenode for HDFS File Metadata (08:40)
18. Recap of HDFS on Dataproc Cluster (11:32)

47. Build Hive Applications in Hadoop and Spark Clusters
1. Introduction to Building Hive Applications (02:11)
2. Getting Started with Hive (15:02)
3. Overview of Hive Architecture (08:38)
4. Overview of Integrating Hive Commands with Shell Scripts (07:16)
5. Run Hive Commands using Scripts (08:52)
6. Override Run time Hive Configuration Properties and Variables (11:09)
7. Getting Started Data Loader for NYSE Data using HIve (07:24)
8. Design NYSE Data Loader Application (05:52)
9. Create Partitioned Parquet Table for NYSE Data (11:30)
10. Populate Data into Partitioned NYSE table from Stage Table (10:03)
11. Process NYSE Data and load into partitioned table (05:45)
12. Develop Hive QL Script to Load NYSE Data (05:20)
13. Run Hive QL Commands using Script for NYSE Loader (07:33)
14. Redesign the Solution using HDFS to stage files (12:50)
15. Validate Hive Application to Convert NYSE Data (09:13)
16. Deploy Hive Application in HDFS (05:48)
17. Develop Shell Wrapper to run Hive Application (07:50)
18. Overview of Scheduling and Crontab (09:37)
19. Schedule Hive Applications using Cron (13:11)
20. Recap of Application Development Life Cycle using Hive (05:07)

48. Getting Started with Spark SQL on Hadoop and Spark Cluster
1. Getting Started with Spark SQL on Hadoop and Spark Cluster (01:45)
2. Getting Started with Data Sets and Spark SQL CLI (05:17)
3. Spark SQL Metastore Architecture (11:14)
4. Overview of Spark Metastore Warehouse Directory (05:14)
5. Launch Spark SQL CLI with Delta Lake Packages (04:58)
6. Populate Data into Delta Lake Tables using Spark SQL (14:31)
7. Run Individual Spark SQL Commands (10:06)
8. Run Spark SQL Scripts using Spark SQL CLI (08:14)

49. Build Real Time Applications using Spark SQL with Shell Wrapper
1. Introduction to Application Development Life Cycle of Spark SQL Applications (01:11)
2. Review the Requirements and Datasets for NYSE Data (05:33)
3. Design NYSE Converter Application using Spark SQL and Delta (04:22)
4. Copy Files into HDFS for NYSE Converter (04:28)
5. Create External Stage Table for NYSE CSV Files (07:30)
6. Create Target Table for NYSE Data using Delta Format (08:44)
7. Populate Data for Additional Years into Delta NYSE Table (15:30)
8. Develop Spark SQL Application for NYSE Data Conversion (05:15)
9. Validate Spark SQL Application for NYSE Data Conversion (06:56)
10. Develop Shell Wrapper for Spark SQL Application (06:54)

50. Getting Started with Pyspark on Hadoop and Spark Cluster
1. Introduction to Getting Started with Pyspark (01:31)
2. Launching or Getting Started Pyspark CLI (09:03)
3. Overview of Spark Properties Files (04:56)
4. Review Data Sets to explore Pyspark APIs (04:34)
5. Getting Started with Pyspark for Data Processing (06:14)
6. Read Orders and Order Items Data into Spark Data Frames (05:28)
7. Process Data using pyspark Dataframe APIs (10:06)
8. Overview of Spark Submit Command (04:09)

51. Submitting Python based Spark Applications
1. Introduction to Submitting Python based Spark Applications (01:03)
2. Develop Pyspark Application for Daily Revenue (05:52)
3. Run Spark Application using spark-submit (06:05)
4. Specify Paths using Environment Variables in Spark Applications (06:26)
5. Run Spark Application with Environment Variables in Client Mode (04:07)
6. Run Spark Application with Environment Variables in Cluster Mode (05:08)
7. Review Spark Application Details using Spark UI (03:11)
8. Review YARN Logs for Spark Applications in Cluster Mode (06:51)
9. Overview of Execution Process of Spark Applications (07:27)
10. Deep Dive into Spark Deploy Modes (13:29)
11. Submit Spark Applications with dependencies as packages (12:10)
12. Submit Spark Applications with dependencies as jars (16:19)
13. Develop Shell Wrappers to submit Spark Applications (07:49)

52. Logging in Python based Spark Applications
1. Introduction to Logging in Python baesd Spark Applications (01:06)
2. Run Application without logging (06:33)
3. Overview of Logging Concepts such as Log Levels (04:50)
4. Getting Started with logging using Python (06:52)
5. Changing the Log Message Format using logging (06:28)
6. Add logging to Python based Spark Applications (03:56)
7. Validate Logging of Spark Application using Client Mode (03:49)
8. Validate Logging of Spark Application using Cluster Mode (05:16)

53. Performance Tuning of Spark Applications on Hadoop and Spark
1. Introduction to Performance Tuning of Spark Applications on Hadoop and Spark Clu (02:03)
2. Delete Single Node Hadoop and Spark Cluster using Dataproc (02:12)
3. Increase GCP VM Quotas for Mutlinode Hadoop and Spark Cluster (03:42)
4. Review Quotas to setup Multinode Hadoop and Spark Cluster (02:17)
5. Setup Multinode Hadoop and Spark Cluster using GCP Dataproc (03:47)
6. Setup SSH Connectivity and VS Code Workspace using Master Node (07:19)
7. Review Multi Node Hadoop and Spark Clusters using Web Interfaces (05:14)
8. Overview of Multinode Hadoop and Spark Cluster Topology (12:08)
9. Computing Overall Capacity of Multinode Hadoop and Spark Clusters (06:06)
10. Determine Overall YARN Capacity (05:24)
11. Overview of Spark History Server UI (04:53)
12. Generate Test Data for Spark Performance Tuning (04:23)
13. Develop Word Count Application using Spark (11:57)
14. Develop and Validate Shell Script for Word Count (05:05)
15. Overview of Jobs related to Spark Applications using Spark UI (08:14)
16. Review Environment Properties and Disabling Dynamic Allocation (07:42)
17. Overriding Spark Executor Instances to tune the performance (07:14)
18. Determine Maximum Capacity to submit a Spark Application (03:35)
19. Overview of Adaptive Query Execution (07:03)
20. Overview of Shuffling - Part 1 (08:51)
21. Overview of Shuffling - Part 2 (12:21)
22. Overview of Spark Application (04:45)
23. Overview of Lazy Evaluation (08:40)
24. Review the code of Word Count Application (03:22)
25. Run Spark Application with out Adaptive Query Execution (10:12)
26. Run Spark Application using Adaptive Query Execution (06:20)
27. Overview of Spark Dynamic Allocation (09:27)
28. Demo on Spark Dynamic Allocation (10:41)
29. Running Spark Application using Dynamic Allocation (06:46)
30. Overview of number of Spark Partitions (08:50)
31. Delete Multinode Hadoop and Spark Cluster (03:09)
[/spoiler]
Файлы примеров: не предусмотрены
Формат видео: MP4
Видео: AVC, 1280x720, 16:9, 30fps, 972kbps
Аудио: AAC, 44.1kHz, 62.8kbps, stereo
[spoiler="Скриншоты"]

[/spoiler]

Related Torrents

torrent name	size	uploader	age	seed	leech
Udemy - Databricks Data Engineering with DLT and Lakeflow Pipelines Posted by freecoursewb in Other	3.1 GB	freecoursewb	2 months	8	3
Udemy - Databricks Data Engineering - Build Pipelines With Lakeflow Posted by freecoursewb in Other	3 GB	freecoursewb	3 months	1	8
Udemy - Snowflake Data Engineering Preparations Course- DEA-C02 Posted by freecoursewb in Other	1.8 GB	freecoursewb	5 months	6	0
Databricks for Data Engineering - PySpark, SQL and Delta Lake Posted by freecoursewb in Other	2.4 GB	freecoursewb	6 months	7	5
SQL for Data Engineering - Real-World Pipeline and Enterprise Posted by freecoursewb in Other	1.2 GB	freecoursewb	7 months	0	0

Data Engineering for Beginners: Learn SQL, Python & Spark [2/2025, ENG]

Files

Description

Related Torrents

Trackers

All Comments

tracker name
udp://tracker.coppersurfer.tk:6969/announce
udp://9.rarbg.me:2850/announce
udp://9.rarbg.to:2920/announce
udp://tracker.opentrackr.org:1337
udp://tracker.leechers-paradise.org:6969/announce