Heyy guys, how are you doing? Hope you all are doing well and are in good health. Today’s lesson is all about web scraping. What is Web Scraping?, How is it useful? Is it an important skill to have?, all of these questions will be answered here. This is just an introduction to what a web scraper/scraping actually means. We will not be learning web scraping today but we will definitely learn how to create web scrapers using python very soon.
What is Web Scraping?
To put it simply, Web Scraping means extracting data from the
web. We are all very much familiar with the web. This article is on the web
and you are reading this from the web. Why do we call this a web? In simple
terms, all of us and all of our devices with an internet connection is part of
the web. We all are connected together because of the web.
The web holds an
enormous amount of data. Imagine, you being capable of accessing all this data
and using it for yourself, all by executing a simple python program. Wouldn’t
that be amazing?? This is where the art of web scraping comes in.
What are Web Scrapers?
A Web Scraper is a program that extracts data from a website. Remember the python program I told you about in the last para that can extract data from the web? That program is a web scraper. For example, let us say you want to know the prices of the top laptops under $500 on amazon. You can create a web scraper that goes to the amazon’s official website, searches “Laptops under $500” and it saves the names and the prices of the top 20 laptops from the results to a text file on your desktop. All of this done in mere seconds. Imagine the time you would save doing this instead of going on to amazon and searching for the laptops yourself like any normal person would do.
Web scrapers can be and
are used by professionals and companies in all fields/industrues. Financial
institutions like Goldman Sachs use web scrapers to get data from the web. It is
essentially what connects a company to the infinite amount of data on the web. Every programmer must know how to extract data
from the web or as we call it ‘Web scraping’. People actually specialise in
this and get a pretty good job in a good company. As a web scraper, you act as
a bridge between the company and the rest of the world through the web. This a
must have skill for a programmer.
How do websites work?
For us to get data from
the web, we must have a basic understanding of a website’s design and how it
works.
You guys might have
heard about HTML, CSS and JavaScript. These are basically the programming languages
used by web developers a.k.a people who design and create websites. If you guys
want to see what a HTML (Hyper Text Markup Language) code looks like, open any
website on your web browser, Press Ctrl + U, that should do the trick most of the times, if that doesn't work, try a Right click on the page and select the ‘Page Source’
option from the dropdown menu. The code that opens up in a new tab is written in HTML. That
is what defines a website, this is the code that is actually running the
website. The website is basically the graphical representation of the HTML
code.
Let us now take a look
at some of the terms used in web scraping:
HTML
tag: A
HTML tag is like a programming keyword that tell our browser to do something. For example, the tag ‘<hr>’ tells our browser to create a horizontal line.
Understanding tags is a very important part of web scraping because you can use
a tag to tell the program that you are referring to a specific piece of code.
Parsing: Parsing means taking a format (like HTML) and using a programming language to give it structure that we can work with. For example, turning HTML code into objects with methods.
CONGRATULATIONS !!! You now know what a web scraper is and have an idea about how HTML is used to design a website. I know this was supposed to be a python lesson, I just wanted you guys to be familiar with this because we will be using the tags in the html code for web scraping.
So, this was an introduction to web scraping. Stay tuned for another article next week, same time, where I will teach you a fun trick to do in Sublime text 3 (My Favourite IDE) that will your life as a python programmer a lot easier. So more cool stuff coming your way, DON’T MISS IT !! And I'll see you next week. Goodbye and Good Luck :)
I hope this article answered all of your questions and even helped you in becoming a better programmer. IF IT DID, leave a like AND FOLLOW THIS BLOG TO BECOME A PROFESSIONAL PYTHON PROGRAMMER FROM A TOTAL BEGINNER. IF IT DIDN'T, feel free to ask any further queries in the comment section below.
HOPE YOU HAVE AN AWESOME DAY AHEAD !!!
2 Comments
That's a really cool lesson on Web scraping brother. Thank you 👍
ReplyDeleteThank you so much for your generous comment. I am really glad it help you :)
DeleteWelcome to the comments section, this is where you can contact me personally for any doubts or feedback