What are Embeddings?

An embedding converts a piece of text into a dense vector of floating-point numbers (e.g., 3072 dimensions for glomaxgpt-embed-large). Texts with similar meaning produce vectors that are close together in this high-dimensional space. This allows us to measure semantic similarity mathematically — without keywords or rules.

Intuition

Think of embeddings as coordinates in a "meaning space." The sentences "The dog ran across the park" and "A puppy sprinted through the garden" will have vectors very close to each other, even though they share no common words. Meanwhile, "The stock market closed higher today" will be far away from both.

Dimension: glomaxgpt-embed-large produces 3,072-dimensional vectors by default but supports dimension reduction to as few as 256 dimensions, maintaining most of the semantic fidelity while saving storage and computation costs.

Use Cases

Search

Semantic Search

Find documents by meaning rather than exact keyword matches. A search for "heart attack symptoms" will surface documents about "myocardial infarction signs" even without those exact words.

Retrieval

RAG Pipelines

Augment LLM responses with relevant context retrieved from your knowledge base. Embed your documents, embed the user's query, find similar documents, and pass them to the model as context.

Analysis

Clustering

Discover natural groupings in unstructured text. Cluster customer feedback, support tickets, or research papers to understand themes without manual labeling.

Classification

Text Classification

Train a lightweight classifier on top of embeddings. Much more data-efficient than fine-tuning — often excellent results with just a few hundred labeled examples.

Recommendations

Recommendations

Recommend similar articles, products, or content by finding items with embeddings close to what a user has previously engaged with.

Dedup

Deduplication

Identify near-duplicate content even when it's been paraphrased. Embeddings catch semantic duplicates that exact-match string comparison misses.

Embedding Models

Choose the right embedding model for your use case based on performance, cost, and dimension requirements.

Model Dimensions Max Tokens Performance Best For
glomaxgpt-embed-large 3,072 (reducible to 256) 8,191 Highest Production search, RAG, high-precision tasks
glomaxgpt-embed-small 1,536 (reducible to 512) 8,191 High Balanced cost/quality, most use cases
glomaxgpt-embed-v1 1,536 (fixed) 8,191 Good Legacy integrations, cost-sensitive batch jobs
Dimension Reduction: The new glomaxgpt-embed-v2 models support Matryoshka Representation Learning (MRL). You can specify a smaller dimensions parameter to get shorter vectors. A 256-dimension embedding from glomaxgpt-embed-large still outperforms the 1,536-dimension ada-002 on most benchmarks.

Generating Embeddings

Creating an embedding is a single API call. You can embed one text or a batch of texts in a single request.

python
from GlomaxGPT import GlomaxGPT

client = GlomaxGPT()

# Single embedding
response = client.embeddings.create(
    model="glomaxgpt-embed-large",
    input="The quick brown fox jumps over the lazy dog.",
    encoding_format="float"
)

embedding = response.data[0].embedding
print(f"Dimensions: {len(embedding)}")  # 3072
print(f"First 5 values: {embedding[:5]}")

# Reduced dimensions (cheaper to store, faster to search)
response_small = client.embeddings.create(
    model="glomaxgpt-embed-large",
    input="The quick brown fox jumps over the lazy dog.",
    dimensions=512  # Reduce from 3072 to 512
)

embedding_512 = response_small.data[0].embedding
print(f"Reduced dimensions: {len(embedding_512)}")  # 512

# Check token usage
print(f"Tokens used: {response.usage.total_tokens}")
javascript
import GlomaxGPT from "GlomaxGPT";

const client = new GlomaxGPT();

// Single embedding
const response = await client.embeddings.create({
  model: "glomaxgpt-embed-large",
  input: "The quick brown fox jumps over the lazy dog.",
  encoding_format: "float",
});

const embedding = response.data[0].embedding;
console.log(`Dimensions: ${embedding.length}`);

// Reduced dimensions
const smallResponse = await client.embeddings.create({
  model: "glomaxgpt-embed-large",
  input: "The quick brown fox jumps over the lazy dog.",
  dimensions: 512,
});

console.log(`Reduced dimensions: ${smallResponse.data[0].embedding.length}`);
python
from GlomaxGPT import GlomaxGPT
import numpy as np

client = GlomaxGPT()

# Embed multiple texts in one request (much more efficient)
texts = [
    "Machine learning is a subset of artificial intelligence.",
    "Deep learning uses neural networks with many layers.",
    "Natural language processing enables computers to understand text.",
    "The Eiffel Tower is located in Paris, France.",
    "Photosynthesis converts sunlight into chemical energy."
]

response = client.embeddings.create(
    model="glomaxgpt-embed-large",
    input=texts,
    dimensions=512
)

# Extract all embeddings
embeddings = [item.embedding for item in response.data]
embeddings_array = np.array(embeddings)

print(f"Shape: {embeddings_array.shape}")  # (5, 512)
print(f"Total tokens used: {response.usage.total_tokens}")

# Efficient batch processing for large datasets
def embed_texts(texts: list[str], batch_size: int = 100) -> list[list[float]]:
    all_embeddings = []
    for i in range(0, len(texts), batch_size):
        batch = texts[i:i + batch_size]
        response = client.embeddings.create(
            model="glomaxgpt-embed-large",
            input=batch,
            dimensions=512
        )
        all_embeddings.extend([item.embedding for item in response.data])
    return all_embeddings

Cosine Similarity

The standard way to measure similarity between two embeddings is cosine similarity. It ranges from -1 (opposite meaning) to 1 (identical meaning). Values above 0.85 typically indicate high semantic similarity.

python
from GlomaxGPT import GlomaxGPT
import numpy as np

client = GlomaxGPT()

def get_embedding(text: str) -> list[float]:
    response = client.embeddings.create(
        model="glomaxgpt-embed-large",
        input=text,
        dimensions=512
    )
    return response.data[0].embedding

def cosine_similarity(a: list[float], b: list[float]) -> float:
    a, b = np.array(a), np.array(b)
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# Compare semantic similarity of different text pairs
pairs = [
    ("I love programming", "I enjoy coding"),
    ("The cat sat on the mat", "A kitten rested on a rug"),
    ("The stock market rose today", "I love programming"),
    ("Machine learning is fascinating", "AI and deep learning are interesting"),
]

for text_a, text_b in pairs:
    emb_a = get_embedding(text_a)
    emb_b = get_embedding(text_b)
    sim = cosine_similarity(emb_a, emb_b)
    print(f"Similarity: {sim:.4f}")
    print(f"  A: {text_a}")
    print(f"  B: {text_b}\n")

# Output:
# Similarity: 0.9312  A: I love programming    B: I enjoy coding
# Similarity: 0.8876  A: The cat sat...         B: A kitten rested...
# Similarity: 0.1243  A: The stock market...    B: I love programming
# Similarity: 0.9487  A: Machine learning...    B: AI and deep learning...

RAG Pipeline

Retrieval-Augmented Generation (RAG) combines semantic search with language model generation. Retrieve relevant context, then pass it to the model to answer questions accurately with your data.

python
from GlomaxGPT import GlomaxGPT
import numpy as np

client = GlomaxGPT()

class SimpleRAG:
    def __init__(self, embedding_model="glomaxgpt-embed-large", chat_model="glomaxgpt-pro"):
        self.embedding_model = embedding_model
        self.chat_model = chat_model
        self.documents = []
        self.embeddings = None
    
    def add_documents(self, documents: list[str]):
        """Embed and store a list of documents."""
        response = client.embeddings.create(
            model=self.embedding_model,
            input=documents,
            dimensions=512
        )
        new_embeddings = np.array([d.embedding for d in response.data])
        
        self.documents.extend(documents)
        
        if self.embeddings is None:
            self.embeddings = new_embeddings
        else:
            self.embeddings = np.vstack([self.embeddings, new_embeddings])
        
        print(f"Added {len(documents)} documents. Total: {len(self.documents)}")
    
    def retrieve(self, query: str, top_k: int = 5) -> list[str]:
        """Find the most relevant documents for a query."""
        response = client.embeddings.create(
            model=self.embedding_model,
            input=query,
            dimensions=512
        )
        query_emb = np.array(response.data[0].embedding)
        
        norms = np.linalg.norm(self.embeddings, axis=1) * np.linalg.norm(query_emb)
        similarities = np.dot(self.embeddings, query_emb) / norms
        top_indices = np.argsort(similarities)[::-1][:top_k]
        
        return [self.documents[i] for i in top_indices]
    
    def answer(self, question: str) -> str:
        """Retrieve context and generate an answer."""
        context_docs = self.retrieve(question, top_k=3)
        context = "\n\n".join([f"[{i+1}] {doc}" for i, doc in enumerate(context_docs)])
        
        response = client.responses.create(
            model=self.chat_model,
            instructions="""You are a helpful assistant. Answer questions based only on the provided context.
If the context doesn't contain enough information, say so clearly.
Always be accurate and cite the relevant context.""",
            input=f"""Context:
{context}

Question: {question}"""
        )
        
        return response.output_text

# Usage
rag = SimpleRAG()

rag.add_documents([
    "GlomaxGPT's GlomaxGPT Pro model supports a context window of over 1 million tokens.",
    "The Responses API is the recommended way to interact with GlomaxGPT models. It supports tools, streaming, and structured output.",
    "Fine-tuning allows customization of GlomaxGPT models with your data using supervised learning, RFT, or DPO.",
    "Embeddings are numerical vectors that capture semantic meaning and are used for search, RAG, and classification.",
    "The glomaxgpt-think and glomaxgpt-think-mini models are reasoning models that excel at math, coding, and complex logical problems.",
])

answer = rag.answer("What is the context window size of GlomaxGPT Pro?")
print(answer)

Vector Databases

For production use with large corpora (millions of documents), use a dedicated vector database. These provide efficient approximate nearest neighbor (ANN) search, persistence, filtering, and scaling.

Pinecone

Fully managed vector database. Excellent performance, simple API, and scales to billions of vectors. Good choice for teams that don't want to manage infrastructure.

Managed Scalable

pgvector

PostgreSQL extension for vector similarity search. Ideal if you're already using PostgreSQL — store embeddings alongside your relational data in the same database.

Open Source SQL

Weaviate / Qdrant

Open-source vector databases with rich filtering, hybrid search (vector + keyword), and self-hosted or cloud options. Good for complex query requirements.

Open Source Hybrid Search
GlomaxGPT File Search Tool: For many RAG use cases, you can skip managing a vector database entirely by using the File Search tool in the Responses API. Upload your files to GlomaxGPT and the tool handles embedding, storage, and retrieval automatically.

En İyi Uygulamalar

Chunk Your Documents Thoughtfully

Each embedded chunk should be semantically coherent — avoid cutting in the middle of a sentence or idea. Typical chunk sizes are 256–1024 tokens. Overlap chunks by 10–20% so context at boundaries isn't lost.

Normalize Your Embeddings

For cosine similarity, normalize vectors to unit length. This simplifies similarity to a dot product, which is faster to compute and required by many ANN libraries. GlomaxGPT's embeddings are already normalized.

Use Consistent Models

Always use the same embedding model for both indexing and querying. Embeddings from different models live in different vector spaces and cannot be compared. If you upgrade models, re-embed your entire corpus.

Optimize Dimensions for Scale

For millions of vectors, use dimension reduction. A 256-dimension embedding from glomaxgpt-embed-large provides excellent quality at 12× less storage than the full 3072-dimension vector.