📸 SnapNote – AI-Powered Handwritten Note Extractor

SnapNote is an intelligent web application that helps users digitize and organize handwritten notes. With just a photo, SnapNote extracts handwritten text using advanced AI models and classifies the content into meaningful categories – making your notes searchable, organized, and easy to manage.

---

🚀 Features

• 📷 Upload Handwritten Notes
  Snap a photo or upload scanned handwritten notes directly through the web app.

• 🧠 Text Extraction with Florence
  Uses Microsoft's state-of-the-art multimodal model microsoft/Florence-2-base to extract text from handwritten images.

• 🏷️ Smart Categorization
  Applies zero-shot classification using facebook/bart-large-mnli to assign relevant tags (e.g., Math, To-Do, Personal, Science, etc.) to the extracted content.

• 🗃️ Organized Output
  Automatically presents and groups your notes under categorized tags.

---

🧠 AI Models Used

1. ✍️ Text Extraction

• Model: microsoft/Florence-2-base
• Task: Image-to-text extraction for digitizing handwritten notes.

2. 🏷️ Zero-Shot Classification

• Model: facebook/bart-large-mnli
• Task: Assigning semantic categories to notes without needing task-specific training.

from transformers import pipeline

zero_shot_classifier = pipeline(
    "zero-shot-classification",
    model="facebook/bart-large-mnli"
)

result = zero_shot_classifier(
    "Finish the physics assignment on Newton’s laws.",
    candidate_labels=["To-Do", "Science", "Math", "Personal"]
)

🖥️ Demo

🎥 Video Demo

Watch our live demo and walkthrough on Drive

🖼️ Interface Preview

---

🛠️ Tech Stack

Frontend

• HTML
• CSS
• JavaScript

Backend

• Python
• Flask (used for API endpoints and model integration)

AI Models

• microsoft/Florence-2-base for handwriting text extraction
• facebook/bart-large-mnli for zero-shot classification

Model Integration

• Hugging Face Transformers
• Served locally via Flask

Storage

• Local file system for storing uploaded images and processed notes

---

📦 Installation

To run SnapNote locally, follow these steps:

1. Clone the repository

git clone https://github.com/Ihssane5/SnapNotes

2. Navigate to the folder

cd snapnote

3. Create and activate the Conda environment

Make sure you have Conda installed. Then run:

conda env create -f environment.yaml -n notes
conda activate notes

This will create and activate a virtual environment named notes using the packages listed in your environment.yaml file.

4. Run the application

Once the environment is activated:

python app.py

The app should now be accessible in your browser at http://localhost:5000.

---

🧩 Example Use Case

1. User uploads a handwritten note image.
2. The app extracts text using Florence.
3. The extracted text is classified into tags like "To-Do", "Science", or "Personal" using BART-MNLI.
4. Notes are displayed and grouped under these categories for easy access.

---

🚧 Future Enhancements

• Improve OCR post-processing to enhance accuracy on low-quality scans.
• Enable custom tag suggestions using user history.
• Build mobile app version with real-time capture.
• Add user authentication and personalized dashboards.

---

👩‍💻 Contributors

• Ihssane Nedjaoui
• Other Contributors

---

📜 License

This project is licensed under the MIT License. See LICENSE for details.

---

📬 Contact

For support or inquiries: ihssanenedjaoui5@gmail.com