Vector Embeddings
Embeddings are numerical representations of text that capture semantic meaning. They power semantic search, retrieval-augmented generation (RAG), clustering, classification, and recommendation systems. This guide covers everything from generating your first embedding to building a production RAG pipeline.
What are Embeddings?
An embedding converts a piece of text into a dense vector of floating-point numbers (e.g., 3072 dimensions for glomaxgpt-embed-large). Texts with similar meaning produce vectors that are close together in this high-dimensional space. This allows us to measure semantic similarity mathematically — without keywords or rules.
Intuition
Think of embeddings as coordinates in a "meaning space." The sentences "The dog ran across the park" and "A puppy sprinted through the garden" will have vectors very close to each other, even though they share no common words. Meanwhile, "The stock market closed higher today" will be far away from both.
glomaxgpt-embed-large produces 3,072-dimensional vectors by default but supports dimension reduction to as few as 256 dimensions, maintaining most of the semantic fidelity while saving storage and computation costs.
Use Cases
Semantic Search
Find documents by meaning rather than exact keyword matches. A search for "heart attack symptoms" will surface documents about "myocardial infarction signs" even without those exact words.
RAG Pipelines
Augment LLM responses with relevant context retrieved from your knowledge base. Embed your documents, embed the user's query, find similar documents, and pass them to the model as context.
Clustering
Discover natural groupings in unstructured text. Cluster customer feedback, support tickets, or research papers to understand themes without manual labeling.
Text Classification
Train a lightweight classifier on top of embeddings. Much more data-efficient than fine-tuning — often excellent results with just a few hundred labeled examples.
Recommendations
Recommend similar articles, products, or content by finding items with embeddings close to what a user has previously engaged with.
Deduplication
Identify near-duplicate content even when it's been paraphrased. Embeddings catch semantic duplicates that exact-match string comparison misses.
Embedding Models
Choose the right embedding model for your use case based on performance, cost, and dimension requirements.
| Model | Dimensions | Max Tokens | Performance | Best For |
|---|---|---|---|---|
| glomaxgpt-embed-large | 3,072 (reducible to 256) | 8,191 | Highest | Production search, RAG, high-precision tasks |
| glomaxgpt-embed-small | 1,536 (reducible to 512) | 8,191 | High | Balanced cost/quality, most use cases |
| glomaxgpt-embed-v1 | 1,536 (fixed) | 8,191 | Good | Legacy integrations, cost-sensitive batch jobs |
glomaxgpt-embed-v2 models support Matryoshka Representation Learning (MRL). You can specify a smaller dimensions parameter to get shorter vectors. A 256-dimension embedding from glomaxgpt-embed-large still outperforms the 1,536-dimension ada-002 on most benchmarks.
Generating Embeddings
Creating an embedding is a single API call. You can embed one text or a batch of texts in a single request.
from GlomaxGPT import GlomaxGPT
client = GlomaxGPT()
# Single embedding
response = client.embeddings.create(
model="glomaxgpt-embed-large",
input="The quick brown fox jumps over the lazy dog.",
encoding_format="float"
)
embedding = response.data[0].embedding
print(f"Dimensions: {len(embedding)}") # 3072
print(f"First 5 values: {embedding[:5]}")
# Reduced dimensions (cheaper to store, faster to search)
response_small = client.embeddings.create(
model="glomaxgpt-embed-large",
input="The quick brown fox jumps over the lazy dog.",
dimensions=512 # Reduce from 3072 to 512
)
embedding_512 = response_small.data[0].embedding
print(f"Reduced dimensions: {len(embedding_512)}") # 512
# Check token usage
print(f"Tokens used: {response.usage.total_tokens}")
import GlomaxGPT from "GlomaxGPT";
const client = new GlomaxGPT();
// Single embedding
const response = await client.embeddings.create({
model: "glomaxgpt-embed-large",
input: "The quick brown fox jumps over the lazy dog.",
encoding_format: "float",
});
const embedding = response.data[0].embedding;
console.log(`Dimensions: ${embedding.length}`);
// Reduced dimensions
const smallResponse = await client.embeddings.create({
model: "glomaxgpt-embed-large",
input: "The quick brown fox jumps over the lazy dog.",
dimensions: 512,
});
console.log(`Reduced dimensions: ${smallResponse.data[0].embedding.length}`);
from GlomaxGPT import GlomaxGPT
import numpy as np
client = GlomaxGPT()
# Embed multiple texts in one request (much more efficient)
texts = [
"Machine learning is a subset of artificial intelligence.",
"Deep learning uses neural networks with many layers.",
"Natural language processing enables computers to understand text.",
"The Eiffel Tower is located in Paris, France.",
"Photosynthesis converts sunlight into chemical energy."
]
response = client.embeddings.create(
model="glomaxgpt-embed-large",
input=texts,
dimensions=512
)
# Extract all embeddings
embeddings = [item.embedding for item in response.data]
embeddings_array = np.array(embeddings)
print(f"Shape: {embeddings_array.shape}") # (5, 512)
print(f"Total tokens used: {response.usage.total_tokens}")
# Efficient batch processing for large datasets
def embed_texts(texts: list[str], batch_size: int = 100) -> list[list[float]]:
all_embeddings = []
for i in range(0, len(texts), batch_size):
batch = texts[i:i + batch_size]
response = client.embeddings.create(
model="glomaxgpt-embed-large",
input=batch,
dimensions=512
)
all_embeddings.extend([item.embedding for item in response.data])
return all_embeddings
Cosine Similarity
The standard way to measure similarity between two embeddings is cosine similarity. It ranges from -1 (opposite meaning) to 1 (identical meaning). Values above 0.85 typically indicate high semantic similarity.
from GlomaxGPT import GlomaxGPT
import numpy as np
client = GlomaxGPT()
def get_embedding(text: str) -> list[float]:
response = client.embeddings.create(
model="glomaxgpt-embed-large",
input=text,
dimensions=512
)
return response.data[0].embedding
def cosine_similarity(a: list[float], b: list[float]) -> float:
a, b = np.array(a), np.array(b)
return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
# Compare semantic similarity of different text pairs
pairs = [
("I love programming", "I enjoy coding"),
("The cat sat on the mat", "A kitten rested on a rug"),
("The stock market rose today", "I love programming"),
("Machine learning is fascinating", "AI and deep learning are interesting"),
]
for text_a, text_b in pairs:
emb_a = get_embedding(text_a)
emb_b = get_embedding(text_b)
sim = cosine_similarity(emb_a, emb_b)
print(f"Similarity: {sim:.4f}")
print(f" A: {text_a}")
print(f" B: {text_b}\n")
# Output:
# Similarity: 0.9312 A: I love programming B: I enjoy coding
# Similarity: 0.8876 A: The cat sat... B: A kitten rested...
# Similarity: 0.1243 A: The stock market... B: I love programming
# Similarity: 0.9487 A: Machine learning... B: AI and deep learning...
Semantic Search
Build a search system that finds documents by semantic meaning. Embed all your documents once, store the vectors, then for each query embed the search term and find the most similar documents.
from GlomaxGPT import GlomaxGPT
import numpy as np
client = GlomaxGPT()
# Sample knowledge base
documents = [
{"id": 1, "text": "GlomaxGPT was founded in 2015 by Sam Altman, Greg Brockman, and others with the mission of ensuring AI benefits all of humanity."},
{"id": 2, "text": "The GPT series of models uses transformer architecture and is pre-trained on large text corpora."},
{"id": 3, "text": "The Responses API supports multi-turn conversations, built-in tools like web search, and structured JSON output."},
{"id": 4, "text": "Fine-tuning allows you to customize GlomaxGPT models with your own training data for improved task performance."},
{"id": 5, "text": "DALL·E and GlomaxGPT Vision 2 are GlomaxGPT's image generation models, capable of creating photorealistic images from text descriptions."},
{"id": 6, "text": "GlomaxGPT Voice STT is GlomaxGPT's speech recognition model, supporting transcription and translation in 99 languages."},
]
# Step 1: Embed all documents (do this once and store)
doc_texts = [d["text"] for d in documents]
embed_response = client.embeddings.create(
model="glomaxgpt-embed-large",
input=doc_texts,
dimensions=512
)
doc_embeddings = np.array([item.embedding for item in embed_response.data])
# Step 2: Search function
def semantic_search(query: str, top_k: int = 3) -> list[dict]:
# Embed the query
query_response = client.embeddings.create(
model="glomaxgpt-embed-large",
input=query,
dimensions=512
)
query_embedding = np.array(query_response.data[0].embedding)
# Compute cosine similarities
norms = np.linalg.norm(doc_embeddings, axis=1) * np.linalg.norm(query_embedding)
similarities = np.dot(doc_embeddings, query_embedding) / norms
# Get top-k results
top_indices = np.argsort(similarities)[::-1][:top_k]
results = []
for idx in top_indices:
results.append({
"document": documents[idx],
"similarity": float(similarities[idx])
})
return results
# Test the search
results = semantic_search("How do I convert speech to text?")
for r in results:
print(f"Score: {r['similarity']:.4f} | {r['document']['text'][:80]}...")
RAG Pipeline
Retrieval-Augmented Generation (RAG) combines semantic search with language model generation. Retrieve relevant context, then pass it to the model to answer questions accurately with your data.
from GlomaxGPT import GlomaxGPT
import numpy as np
client = GlomaxGPT()
class SimpleRAG:
def __init__(self, embedding_model="glomaxgpt-embed-large", chat_model="glomaxgpt-pro"):
self.embedding_model = embedding_model
self.chat_model = chat_model
self.documents = []
self.embeddings = None
def add_documents(self, documents: list[str]):
"""Embed and store a list of documents."""
response = client.embeddings.create(
model=self.embedding_model,
input=documents,
dimensions=512
)
new_embeddings = np.array([d.embedding for d in response.data])
self.documents.extend(documents)
if self.embeddings is None:
self.embeddings = new_embeddings
else:
self.embeddings = np.vstack([self.embeddings, new_embeddings])
print(f"Added {len(documents)} documents. Total: {len(self.documents)}")
def retrieve(self, query: str, top_k: int = 5) -> list[str]:
"""Find the most relevant documents for a query."""
response = client.embeddings.create(
model=self.embedding_model,
input=query,
dimensions=512
)
query_emb = np.array(response.data[0].embedding)
norms = np.linalg.norm(self.embeddings, axis=1) * np.linalg.norm(query_emb)
similarities = np.dot(self.embeddings, query_emb) / norms
top_indices = np.argsort(similarities)[::-1][:top_k]
return [self.documents[i] for i in top_indices]
def answer(self, question: str) -> str:
"""Retrieve context and generate an answer."""
context_docs = self.retrieve(question, top_k=3)
context = "\n\n".join([f"[{i+1}] {doc}" for i, doc in enumerate(context_docs)])
response = client.responses.create(
model=self.chat_model,
instructions="""You are a helpful assistant. Answer questions based only on the provided context.
If the context doesn't contain enough information, say so clearly.
Always be accurate and cite the relevant context.""",
input=f"""Context:
{context}
Question: {question}"""
)
return response.output_text
# Usage
rag = SimpleRAG()
rag.add_documents([
"GlomaxGPT's GlomaxGPT Pro model supports a context window of over 1 million tokens.",
"The Responses API is the recommended way to interact with GlomaxGPT models. It supports tools, streaming, and structured output.",
"Fine-tuning allows customization of GlomaxGPT models with your data using supervised learning, RFT, or DPO.",
"Embeddings are numerical vectors that capture semantic meaning and are used for search, RAG, and classification.",
"The glomaxgpt-think and glomaxgpt-think-mini models are reasoning models that excel at math, coding, and complex logical problems.",
])
answer = rag.answer("What is the context window size of GlomaxGPT Pro?")
print(answer)
Vector Databases
For production use with large corpora (millions of documents), use a dedicated vector database. These provide efficient approximate nearest neighbor (ANN) search, persistence, filtering, and scaling.
Pinecone
Fully managed vector database. Excellent performance, simple API, and scales to billions of vectors. Good choice for teams that don't want to manage infrastructure.
pgvector
PostgreSQL extension for vector similarity search. Ideal if you're already using PostgreSQL — store embeddings alongside your relational data in the same database.
Weaviate / Qdrant
Open-source vector databases with rich filtering, hybrid search (vector + keyword), and self-hosted or cloud options. Good for complex query requirements.
En İyi Uygulamalar
Chunk Your Documents Thoughtfully
Each embedded chunk should be semantically coherent — avoid cutting in the middle of a sentence or idea. Typical chunk sizes are 256–1024 tokens. Overlap chunks by 10–20% so context at boundaries isn't lost.
Normalize Your Embeddings
For cosine similarity, normalize vectors to unit length. This simplifies similarity to a dot product, which is faster to compute and required by many ANN libraries. GlomaxGPT's embeddings are already normalized.
Use Consistent Models
Always use the same embedding model for both indexing and querying. Embeddings from different models live in different vector spaces and cannot be compared. If you upgrade models, re-embed your entire corpus.
Optimize Dimensions for Scale
For millions of vectors, use dimension reduction. A 256-dimension embedding from glomaxgpt-embed-large provides excellent quality at 12× less storage than the full 3072-dimension vector.