AI Video Search Engine
The AI Video Search Engine is a tool designed to simplify and streamline the process of searching within video content. By integrating natu...
Last verified:
What is AI Video Search Engine?
AI Video Search Engine (AVSE) is an open-source video search engine powered by AI that indexes thousands of videos from YouTube and allows users to search videos by asking questions. The tool uses semantic search and vector embedding technology to find relevant sections of videos based on user queries, providing fast and accurate answers from video content rather than just relying on titles, descriptions, or tags.
Key features include: searching videos with questions to get related answers from within the video, AI-powered video chat with any video after email/OTP registration, video summary generation in both point form and text format with multiple voice reading options, timestamped transcript playback to jump to relevant video clips, and support for over 75+ languages with an 8000+ context size embedding model. The tool also offers a Summary button that displays condensed video content.
AVSE is designed for anyone who wants to find specific answers within millions of videos quickly - including students researching topics, professionals looking for specific tutorial segments, researchers analyzing video content, and casual viewers who want to jump to relevant parts of long videos without watching the entire content. It addresses the gap where traditional YouTube search doesn't index transcribed audio content.
AI Video Search Engine pricing
Pricing model: Free
The tool appears to be free to use for basic video search functionality. The chat with video feature requires email registration with OTP verification but no mentioned payment. For developers who want to host it themselves, paid Supabase and Fly.io accounts are needed if planning to index thousands to millions of videos. The project is open-source on GitHub with no mentioned commercial pricing tiers for end users.
AI Video Search Engine pros
- Searches videos by asking questions instead of just keywords
- Finds specific sections within videos using semantic search
- Provides AI-generated summaries in point form and text format
- Chat with video feature for deeper content understanding
- Multiple voice reading options for video summaries
- Supports over 75+ languages for international users
- Timestamped transcripts jump to relevant video clips instantly
- Indexes thousands of YouTube videos automatically
- Open-source project with 600+ GitHub stars
- Uses advanced vector cosine search for accurate results
- Temporary chat sessions managed by JigsawStack Prompt Engine
- Larger 8000+ context size embedding model for better retrieval
- No storage of chat sessions for privacy
- Can handle millions of videos with current setup
- Faster than manually watching entire videos to find answers
AI Video Search Engine cons
- Requires email registration and OTP verification for chat feature
- English searches are more accurate than other languages
- Chinese search support is less友好 (not as good)
- Query speed needs significant improvement according to developers
- Lacks traditional search engine ranking and relevance capabilities
- Cannot exclude or include specific channels in search results
- No dark mode option available
- Plans to add TikTok but not yet implemented
- May not handle billions/trillions of videos without more resources
- Paid Supabase and Fly.io accounts needed for hosting thousands to millions of videos
Frequently asked questions about AI Video Search Engine
Doesn't YouTube do this already?
Not really. YouTube doesn't search the transcribed audio of videos but instead relies on written content from the uploader such as title, description, and tags. All audio content goes un-indexed by YouTube, while AVSE indexes the actual video transcription.
How will this stack handle millions of videos?
It can handle millions of videos but maybe not billions or trillions with the current setup. Handling larger volumes would require more replicas, instances, and additional infrastructure resources.
What happened to the previous version?
The previous version used the gte-small embedding model which had a small context length and only supported English with a lower retrieval score. The current version uses a newer model supporting over 75+ languages with a larger context size of 8000+ allowing for smaller embedding outputs and a smaller database.
How does AVSE store videos?
Video transcription is extracted from YouTube videos, then elements like thumbnail, description, and title are extracted. The transcription and data are passed to JigsawStack's multimodal embedding model to generate vector embeddings. Transcriptions are chunked with timestamps and stored in PostgresDB (Supabase) using the pg_vector extension for indexing.
How does the searching process work?
Vector cosine search is performed across the database based on the question to return relevant results. Each video in the result performs a second search to find related video chunks to the question. These chunks are mapped back to the timestamped transcript to play the relevant clip of the video.
How does the summary and chat feature work?
The video transcription is sent to JigsawStack API for summary generation in both point form and text format. Chat sessions are temporary and aren't stored, which is managed by JigsawStack's Prompt Engine API for privacy.
Can I get v1 embeddings?
Yes. The entire DB snapshot of the index and index_embedding table from v1 has been dumped and is available at: https://huggingface.co/datasets/yoeven/ai-video-searchengine-archive
What tech stack does AVSE use?
AVSE uses Supabase (PostgreSQL, PG_Vector, Auth), Hasura (GraphQL layer with permissions), Fly (Hosting of Hasura), JigsawStack (Summary AI, Prompt engine AI, Embedding model), and Vercel (NextJS hosting and Serverless functions).
What's next for AVSE development?
The planned features include adding TikTok as a video source and significantly improving query speed. The developer is open to feedback and continues improving the engine.
Why was AVSE built?
With the rise of short-form content on TikTok and YouTube, more knowledge exists in videos than ever before. AVSE was built to showcase that finding specific answers within millions of videos is possible using readily available technology and infrastructure, similar to how Google indexes text on websites.