Programming + Data
Write a web scraper for public data and analyse it
Collect a table of public information from a site that allows it and analyse the result.
- DifficultyIntermediate
- Estimated time1–2 weeks
- Budget₹0
- ClassClass 11 to college
- TeamSolo
- You needComputer only
Prerequisites: None beyond your class work
Original · by TechnifyedData AnalysisSoftware ProjectFree to doComputer onlyResearch orientedTwo subjects
Overview
Collect a table of public information from a site that allows it and analyse the result.
Skills you will use
Safety
- This project handles personal data. Collect the minimum, store it safely, get permission, and delete it when the project ends.
Materials and tools
- Computer with Python 3
- A site whose terms and robots.txt allow automated access
Tools and software
Step-by-step roadmap
Start
You will finish with: a dataset, the script that produced it and three findings.
- 1
Prerequisites
Nothing special to know first. Then collect the 2 items in the materials list.
- 2
Learn
Read up on python: ethics, HTML parsing, public-data. Your textbook chapter and one short video are enough.
- 3
Plan
List the features for version one, sketch the screens or data flow and pick the tools. Keep the first version small.
- 4
Build
- Check the site's robots.txt and terms, and prefer an official API or data download if one exists.
- Fetch one page and extract the fields you need.
- Loop over pages slowly, with a delay between requests.
- Save the data as a CSV file.
- Analyse and chart it.
- 5
Test
Try it with wrong, empty and unusual input, and ask someone else to use it without your help.
- 6
Document
Write a README: what it does, how to run it, screenshots and what you learned. Report structure
- 7
Present
Show a live demo of one complete task, then the design and the hardest problem you solved. Presentation structure
- 8
Publish
Put the code on GitHub with the README and, if you can, deploy a live demo.
Expected outcome
A dataset, the script that produced it and three findings.
Other versions of this project
Advanced version
Schedule it to run daily and track changes over time.
Ways to do this project
Text marked "You write this" is a prompt for your own work; everything else is specific to this project.
- Research question
- You write thisOne question you can answer with data, narrow enough to finish in the time you have.
- Hypothesis
- You write thisWhat you expect to find, and why.
- Variables
- You write thisWhat you change or compare, what you measure, and what you hold constant.
- Methodology
- You write thisDescribe the method in enough detail that someone else could repeat it: sample, tools, procedure.
- Data collection
- You write thisSay what you will record, how many times, and how you will keep the records safe.
- Analysis
- You write thisChoose the table, chart or test that answers the question; report averages with their spread.
- Limitations
- You write thisList what could have affected the result: small sample, instrument limits, things you could not control.
Literature review
Find five to eight sources (your textbook, review articles, reports) and note what each says about your question. Group them by idea, not one after another, and end with what is still not known.
References
List every source you used in one style throughout (author, year, title, where it was published, link and the date you opened it).
Use this as the outline of your README or case study.
- Problem
- You write thisThe problem in one or two sentences.
- Why it matters
- You write thisWho has this problem and what it costs them.
- What was built
- A dataset, the script that produced it and three findings.
- Technology
- Python
- Architecture
- You write thisOne diagram: the parts and how data moves between them.
- Implementation
- You write thisThe 5 build steps in your own words, with one code or design decision you are proud of.
- Challenges
- You write thisThe hardest bug or decision, and how you got past it.
- Results
- You write thisNumbers: accuracy, speed, users, survey answers.
- Demo and code
- You write thisA link to a live demo or a short video, and to the GitHub repository.
- Lessons learned
- You write thisWhat you would do differently next time.
- Future improvements
- Schedule it to run daily and track changes over time.
- Problem
- Collect a table of public information from a site that allows it and analyse the result.
- Target user
- You write thisOne specific person: who they are and when they would use this.
- Solution
- Collect a table of public information from a site that allows it and analyse the result.
- First version (MVP)
- You write thisThe smallest version that shows the idea working once, end to end.
- Tech stack
- Python
- Architecture
- You write thisA quick sketch of the parts and how they connect.
- Demo
- You write thisThe one scenario you will show live, start to finish, in under two minutes.
- Stretch goals
- Schedule it to run daily and track changes over time.
Team roles
Plan by length
24-hour
- Hours 0–2: agree the problem, the user and the one thing the demo must show
- Hours 2–16: build only the first version
- Hours 16–20: test the demo path end to end and fix what breaks
- Hours 20–24: slides, a 2-minute demo script and a backup recording
48-hour
- Evening 1: problem, user, sketch and task split
- Day 1: first version working end to end
- Day 2 morning: one stretch goal and testing
- Day 2 afternoon: polish, slides and demo practice
1-week
- Day 1: research the problem and talk to two possible users
- Days 2–4: first version
- Day 5: stretch goals
- Day 6: testing and write-up
- Day 7: demo video and presentation
Report and presentation
Report structure
- Title and summary
- Problem and target user
- Features
- Tools and technology
- Design: architecture, data model or screens
- Implementation
- Testing
- Results and screenshots
- Challenges
- Future improvements
- References
Presentation structure
- The problem
- Who it is for
- Live demo
- How it is built (one diagram)
- The hardest part
- Testing and results
- What you learned
- What comes next
Viva questions
Prepare your own answers to these before you present.
- What is the aim of your project, in one sentence?
- What is the principle behind it?
- Why did you choose these materials or tools?
- What result did you get, and was it what you expected?
- What went wrong and how did you fix it?
- How could this be improved or used in real life?
Resources
Source and attribution
Original
Written by Technifyed. Free to use for your own school or college project; write the report in your own words. Added 1 Oct 2026.
Similar projects
More Programming projects
Programming + Cybersecurity
Make a file encryption tool
Encrypt and decrypt files with a password using a well-tested library.
Programming · Python
Build a Markdown-to-HTML converter
Write a tool that converts headings, lists, links, bold and code blocks from Markdown to HTML.
Programming · Python
Create a text adventure game engine
Build a game where rooms, items and puzzles are loaded from a data file.
Programming · Python
Build a Snake game
Program the classic Snake with scoring, growing length and game over.
Same tools
Physics + Programming
Simulate projectile motion with air drag in Python
Plot real trajectories with air resistance and compare them with the textbook parabola.
Artificial Intelligence · Deep Learning
Classify handwritten digits with a neural network
Train a small neural network on the MNIST digits and study where it goes wrong.
Artificial Intelligence · NLP
Detect spam messages with a text classifier
Train a model on a public SMS dataset to separate spam from normal messages.
Literature and English + Data
Analyse word use in a novel with a computer
Count words and character mentions in a public-domain novel and see what the numbers reveal.
Two-subject alternatives
Music, Theatre and Writing + Programming
Create generative art with code
Write programs that draw a different artwork every time they run and curate a series of ten.
EdTech and Study Tools + Software Development
Build a flashcard app with spaced repetition
Make a web app that shows each card again just before you would forget it.
Graphic and UI/UX Design + Software Development
Redesign an app you use every day
Find real problems in an app through user tests, redesign one flow and prove your version is better.
Environmental Science + Data
Analyse 50 years of your city's temperature data
Download long-term temperature records and test whether your city has warmed, and by how much.
Same difficulty and budget
Robotics, IoT and Hardware + Mechanical Engineering
Design a mechanical part in CAD and simulate its stress
Model a bracket, apply a load in simulation and redesign it to be lighter and still strong.
Software Development · Mobile Apps
Build a habit tracker with streaks
Create an Android app to tick off daily habits and see streaks and a calendar heat map.
Data + Operations, HR and Supply Chain
Build a sales dashboard
Turn a public retail dataset into a one-page dashboard that answers a manager's questions.
Artificial Intelligence + Marketing
Cluster customers into groups
Group customers by spending behaviour with k-means and describe each group in plain words.