Our product (website) provides a solution to tackle the challenge of distance learning between student and teacher, especially in the pandemic era where offline learning was a thing of the past.
Our solution tries to bridge the knowledge gap between a teacher and a student in online classes by leveraging the power of AI to provide the teacher and student with insights to better prepare themselves for the course/lecture, leading to our solution - Insight-Ed.
Client: Tailwind CSS, TypeScript, Next.js
Backend: Flask, Python, Langchain, GeminiPro and GeminiProVision API, OpenAI Whisper API, HuggingFace Open-Source Models (EmotionLLM), MTCNN, Next.js, TypeScript
Storage: PostgreSQL, ChromaDB
Other Tools: GCP (Cloud Run), Docker, Vercel
- Detection of Emotion and Attentiveness of a Student
Used Transfer Learning on EfficientNetB7 model along with a custom dataset combined with an open-sourced LLMs to capture the same through the recorded lecture which the student attended through the webcam during that session.
- Topic Modelling for Each Segment where Student lost attention
-> Through the power of Gemini-Pro SDK and OpenAI Whisper module, each of the segments where the loss of interest is detected is transcribed into text and the most relevant keywords(Topic Modelling) are done to bring out the topics where students need clarification. -> The teacher is thus present with an analysis as to when the student lost his/her attention and on which topic(s) through the video lecture
- Generation of Different Topics on specified time intervals- Saving Teacher's Time
-> This feature allows automatic Generation of Different Topics-Topic Modelling depending on a hyperparameter(time interval) chosen by the teacher exclusively.
- Reverse Video Search
-> This feature allows the student to not get through the video again and again for a particular keyword but rather type it and get the exact timestamp(s) for the same.
- Generation of Questionnaire on topics where students lacked attention
-> This feature allows the teacher to generate/upload a questionnaire through an LLM for the topics for the student to double-check his/her understanding to plan further steps. -> The student can also get recommended most relevant YouTube videos on the same or the teacher could suggest some materials or videos from their course itself.
-
Advance RAG-based QnA bot (Handwritten/Non-Handwritten) -> This feature allows the user to perform question-answer with their uploaded PDFs whether it's Handwritten/Non-Handwritten (under-development). -> This allows the student to quickly search and get relevant information/explanation for a particular topic of interest and not go through searching for it, wasting their precious time.
-
Advance RAG-based QnA bot for Video/Tutorial
-> This feature allows the user to perform question-answer with their video/tutorials. -> This allows the student to quickly search and get relevant information/explanation for a particular topic of interest in the video/tutorial and not go through searching for it throughout the video.
Docker for DesktopPull the server image from DockerHub
docker pull yuvrajsingh9886/insight-ed-server:v1.0Create an environment variable 'DATABASE_URL' in the .env file for the database to work
DATABASE_URL="postgresql://swataswayamdash:[email protected]/Insight-Ed?sslmode=require"Create an environment variable 'GOOGLE_API_KEY' in the .env file for GeminiPro to work
GOOGLE_API_KEY = "YOUR_SECRET_KEY"Run the server image
docker container run -d -p 8080:8080 yuvrajsingh9886/insight-ed-server:v1.0Clone the project
git clone https://github.com/swataswayam-14/Insight-EdGo to the project directory
cd Insight-Ed