Had a call with a startup last week. "Our search is really slow, we are thinking about implementing AI-powered semantic search with vector embeddings."
How many records? "About fifty thousand."
Did you add an index? They hadn't looked at the query plan yet.
Database indexes have existed since the 1970s. The right index can take that search from a full table scan to an index lookup.
"But AI-powered search would understand user INTENT."
Their users search for product names. To match "blue shirt" to a product_name column that contains "Blue Shirt", a trigram index (pg_trgm with GIN) and ILIKE, or Postgres full-text search, solves it. You do not need a transformer model for that.
The same goes for recommendations nobody tried sorting by purchase count, and for analytics running SELECT STAR on a 10 million row table without a WHERE clause.
Embeddings make sense once users search with synonyms or describe what they want instead of typing the product name, and you have measured that keyword search misses those cases. Then hybrid search, keywords plus embeddings, is the next step.
Before reaching for models, embeddings, or a pipeline, look at the query plan. Avoiding a full table scan might be enough. Pagination and caching are worth trying too. For this search, I would start with the index.