May 29, 2020
Back for good
Happy to learn math again.
This month, I learn to do a full-stack data engineer with a goal in mind that is to develop and deploy a data science project. I break it down into smaller steps:
- Build a web app to display some charts based on demo data. I use the Streamlit framework to do it.
https://docs.streamlit.io/en/latest/getting_started.html
- streamlit run app.py - Apply to the company’s database: read an exported CSV file, clean data, and build basic charts. (My teammate gave me the CSV file)
https://towardsdatascience.com/streamlit-101-an-in-depth-introduction-fc8aad9492f2 - Setup Docker to be a localhost server to connect with the Postgres database managed by pgAdmin app. Write SQL queries to explore the database. (My team helped me to setup Docker and database for localhost)
https://www.w3schools.com/sql
- Install Docker, have a file docker-compose.yml
- Terminal: docker -compose up
- Open properties of the database (PyCharm) to fix user/password (if nee)
- Open pgAdmin to restart server. - Connect to the cloud database.
https://docs.sqlalchemy.org/en/13/core/engines.html - Deploy source code to the Azure server.
https://towardsdatascience.com/deploying-a-streamlit-web-app-with-azure-app-service-1f09a2159743
So this month ends with a go-live project that I have learned a lot along the way. The great thing is now I can apply the data science knowledge that I studied before to the real project for the company. Next month I will start to study Deep Learning, which is the higher-level — intermediate, compared to Machine Learning — beginner. Excited to think about it. I am pretty sure that the feeling of doing Data Science, ML/DL will be as interesting as the feeling of doing maths that I enjoyed so much back then.
Once again, it’s cool to look at the little thing that was done :)
Houze HQ, 29 May 2020.
