ShengJun – Medium

Published in Data Arena

·Pinned

Automate Data Cleaning & Combining of Files

Data preparation takes around 80% of any data practitioner’s job (for most instances). Part of the work involves consolidating data across various data files into a single repository for data ingestion. Some of these processes could be automated and here, the notebook showcases several examples to go about it. …

Data Cleaning With Python

8 min read

Automate Data Cleaning & Combining of Files

Published in MLearning.ai

·Pinned

Feature selection — A look at some of the common techniques

Introduction — Imagine you are going to build a model using X number of features to perform a task to a reasonable degree of finesse. With a myriad of methods available, how do you know what number of features to use, and which features to use? Well, one would not be remiss…

Feature Selection

11 min read

Published in Towards Data Science

·Pinned

Overview on apply, map, applymap, iterrows & itertuples

A practical guide with examples — Data cleaning and feature engineering are essential in working with data. I have recently reviewed some past personal projects and refactored them or continued to build upon them. One of these is the visualization of iaito order details. An iaito is an imitation of the katana for iaido study. The…

Data Cleaning

7 min read

Overview on apply, map, applymap, iterrows & itertuples

Published in Data Arena

·Pinned

Web scraping using Python (II)

Guide to web scraping using python & selenium — CONTENT What is Selenium? Why Selenium? Project Pre-requisites Website Code Snippets Data Visualization What is Selenium? Selenium is an open-source web-based automation tool. Originally developed in ThoughtWorks as an in-house tool in 2004, it was eventually released as an open-source. While Selenium is understood as a testing tool, some working on…

Web Scraping

7 min read

Published in Towards Data Science

·Pinned

Data Visualization using Matplotlib & Seaborn

A look at customizing options towards enhancing visualizations, and checking out some lesser-known plot functions along the way — Data visualization provides a visual context through maps or graphs. In doing so, it translates the data to a more natural form for the human mind to comprehend and pick out patterns or points of interest. …

Data Visualization

10 min read

Published in Data Arena

·Aug 1

Speech Analysis Project Series

Part 1 of 3-part series Having recently watched a Netflix documentary “A Life on Our Planet”, I began to wonder about David Attenborough’s speeches regarding the environment. Specifically, how could the speeches be measured? And what can these measures tell us? …

Webscrapp

3 min read

Published in Towards Data Science

·Jul 25

Tips on Pandas, RegEx & Collections

A practical guide with examples — In the previous post related to data cleaning and feature engineering, we went through a couple of operation (apply, map, etc.) examples and their simulated runtimes. In this article, I’ll share other examples that have helped in my work, and hopefully, could help yours too. We’ll be going through examples…

Data Cleaning

6 min read

Published in Data Arena

·Jul 1

Text Analytics with Network graph

Part three of three-part mini-series — Recap In parts one and two of the mini-series, we explored text analytics techniques ranging from word frequency to readability scores to word clouds and sentiment analyses. In this article, we explore the plotting of the story characters’ relationships using network graphs to discern their interactions as the story progresses. Sentiment Analysis with Vader Part two of three-part mini-seriesmedium.com

Network Graph

4 min read

Published in Data Arena

·Jun 2

Sentiment Analysis with Vader

Part two of three-part mini-series — Recap In part one, we performed text analysis by deriving metrics and visualizing them. We also utilized textstat to investigate the readability of the story and did some more visualization using wordclouds. Text Analysis Project Part1 First of three-part mini-project seriesshengjunang.medium.com In this article, we continue with the dataset acquired from bookshare.org and explore Sentiment analysis. Sentiment analysis is a…

Vader

4 min read

Published in Data Arena

·May 13

Text Analysis using Textstat & Wordcloud

Part one of three-part mini-series — Introduction Text analytics can be very fascinating. In this three-part mini-project series, we’ll explore several common text analytics techniques. The dataset is acquired free from bookshare.org. The story I opted to use is ‘Once and Future King’ by T. H. White. There’s a fair bit of data conversion from the .docx…

Word Cloud

7 min read