hero

Kurt Maglalang

Hardware & Software Engineer

Design, Develop, Deliver: Full-Stack and Hardware Solutions for Tomorrow

NetScraper - Web Scrapper

NetScraper - Web Scrapper

Tech Used

Next.JS
Tailwind CSS
Daisy UI
React.JS
MongoDB
TypeScript

Description

NetScraper - Web Scrapper

NetScraper is a cutting-edge web scraping solution developed using Next.js, tailored for efficient and dynamic data extraction from various web sources. This project leverages the power of modern web technologies to provide a seamless and user-friendly experience for scraping data, coupled with robust back-end management using MongoDB and Mongoose.

Description

Key Features:

  • Dynamic Web Scraping: Utilizes advanced techniques to scrape data from multiple web sources efficiently.
  • User-Friendly Interface: Built with Tailwind CSS and DaisyUI for an intuitive and responsive user experience.
  • Data Management: Robust back-end architecture using MongoDB and Mongoose for effective data storage and retrieval.
  • Customizable Scraping: Users can specify parameters for scraping, allowing for targeted data extraction.
  • Real-Time Notifications: Implements React Hot Toast for immediate feedback during web scraping operations.
  • Email Integration: Features Nodemailer for sending notifications or reports via email.
  • Comprehensive Date Handling: Utilizes Moment.js for handling and displaying dates and times effectively.
  • Responsive Design: Ensures a seamless experience across different devices and screen sizes.

Technologies Used:

  • Next.js (v14.0.3): A React framework for building server-rendered applications.
  • Tailwind CSS (v3.3.0): A utility-first CSS framework for rapidly building custom designs.
  • DaisyUI (v4.4.19): Tailwind CSS component library for building beautiful UIs effortlessly.
  • React (v18): A JavaScript library for building user interfaces.
  • Heroicons: A set of free MIT-licensed high-quality SVG icons.
  • MongoDB: NoSQL database for storing scraped data.
  • Mongoose (v8.0.3): MongoDB object modeling for Node.js.
  • Cheerio (v1.0.0-rc.12): Fast, flexible & lean implementation of core jQuery designed specifically for the server.
  • Axios (v1.6.2): Promise-based HTTP client for making HTTP requests.
  • Nodemailer (v6.9.7): Module for sending emails.
  • Moment.js (v2.29.4): Parse, validate, manipulate, and display dates and times in JavaScript.
  • React Hot Toast (v2.4.1): Smoking hot React notifications.
  • ESLint (v8): Pluggable JavaScript linter.
  • Prettier (v3.1.0): Code formatter.
  • Husky (v8.0.3): Modern native Git hooks made easy.
  • Cypress (v13.6.1): End-to-end testing framework.
  • TypeScript (v5): Typed superset of JavaScript that compiles to plain JavaScript.