Numazu's Data Scraper
A web application designed to provide real-time alerts on life-threatening weather and geological events in Numazu-shi, Japan, by scraping and presenting essential updates from the Japan Meteorological Agency (JMA).
Project Links
Please note that the project has not yet launched. Live access will remain unavailable until the official release.
Issues
Even though Japan’s Meteorological Agency (JMA) provides a wealth of information on disasters like earthquakes, tsunamis, and typhoons, I found it incredibly difficult to actually use that information in urgent situations. The site was packed with too many features and too much content, making it hard to extract just the critical info I needed—fast. In emergencies, every second counts, and navigating through multiple layers of maps and pages just to find out if my friends and family in Numazu-shi were safe wasn’t practical. I needed something simpler, faster, and focused. But no existing tools gave me a quick way to check the essentials—especially when stress levels were high and time was limited.
Solutions
I decided to solve this problem by building my own tool tailored specifically to Numazu-shi’s needs. The scraper lets me:
- Instantly retrieve the most important disaster alerts from the JMA website
- Save the data as easy-to-share .txt files and high-res .png maps
- Quickly send critical updates to friends and family in Numazu
- Cut through all the noise, giving me only the info I need in urgent situations
Rather than relying on complicated, inefficient platforms, this system puts the power in my hands—ensuring I get the right data fast, without the stress and confusion. It’s all about providing peace of mind when time is of the essence.
Tech Stack (Fullstack)
- HTML
- CSS
- Javascript
- Node JS
- Express
- Dotenv
- Puppeteer
Tech Stack (DevOps)
- Docker
- Dockerhub
- Portainer
- Ubuntu
- Render
- Cloudlflare
- PM2
Key Features and Technical Highlights
Targeted Disaster Data Extraction with Puppeteer
- Built a real-time web scraper using Puppeteer to extract critical alerts such as earthquakes, tsunamis, and typhoons from the JMA website.
- Filters out non-essential content to deliver fast, focused updates that reduce user overwhelm during emergencies.
High-Resolution Map Rendering
- Automatically captures detailed map visuals and stores them as high-quality .png files.
- Helps users quickly understand affected areas spatially, aiding in faster and more informed decision-making.
Private Alert Distribution Workflow
- Created a lightweight distribution flow that organizes updates into shareable formats.
- Enables immediate private sharing with friends and family in Numazu-shi, ensuring relevant information reaches them quickly.
Hybrid Deployment Approach in Malaysia for Cost Efficiency
- Frontend deployed on Cloudflare Pages for free, fast static hosting, while backend runs on Render and a self-hosted Linux server.
- Achieves a balance of affordability, reliability, and performance, especially during low backend activity.
Custom DevOps Infrastructure with Docker & PM2
- Containerized the system with Docker, managed using PM2 for persistent uptime.
- Used Proxmox, Cloudflare Tunnels, and Portainer.io to maintain secure and efficient remote deployments.
Impact and Outcomes
Reduced Response Time During Emergencies
- Eliminated the need to navigate dense websites manually, providing access to life-saving information within seconds.
- Accelerated awareness and action during natural disasters, when every moment counts.
Improved Safety & Awareness for the Numazu Community
- Delivered localized, simplified alerts to friends and family, boosting their preparedness and peace of mind.
- Made disaster information more accessible, especially during high-pressure situations.
Practical Cost Management with Scalable Hosting
- Avoided unnecessary cloud spend with a hybrid deployment strategy tailored for real-world usage and uptime needs.
- Reflected strong full-stack and DevOps decision-making, prioritizing scalability and sustainability.
Real-World Use, Real Peace of Mind
- This tool wasn’t just theoretical—it’s actively used to safeguard lives and improve readiness.
- Demonstrates initiative and purpose-driven engineering, solving problems that truly matter.