Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,9 @@ From the repo root:
```
For production (Linux only, with Gunicorn for scaling):
```bash
gunicorn app:app --workers 16 --worker-class uvicorn.workers.UvicornWorker --timeout 600 --bind 0.0.0.0:8088
gunicorn app:app --workers 16 --worker-class uvicorn.workers.UvicornWorker --timeout 600 --bind 0.0.0.0:8088
```
- **Note**: For Windows, Gunicorn is not natively supported; use Uvicorn directly with a process manager like PM2 (via Node.js) or NSSM for production scaling.

### Option 2: Run with Podman (Containerized Deployment)

Expand All @@ -36,9 +37,9 @@ From the repo root:
3. Build and run:
```bash
podman build -t oci_genai_gateway .
podman run -p 8088:8088 \
-v ~/.oci:/root/.oci:Z \
-it --name oci_genai_gateway oci_genai_gateway
podman run -p 8088:8088 \
-v ~/.oci:/root/.oci:Z \ # On Linux/macOS; on Windows, use an absolute path like -v C:/Users/<username>/.oci:/root/.oci:Z (ensure the directory is shared/mounted in Podman machine/VM)
-it --name oci_genai_gateway oci_genai_gateway
```
4. Verify: Open `http://localhost:8088` in a browser (should show a health check or API docs). Check logs with `podman logs oci_genai_gateway`.

Expand All @@ -51,7 +52,7 @@ In n8n, use the **OpenAI** node but point it to your local gateway as a custom e
2. Add an **OpenAI** node (under AI > Chat Models).
3. Configure credentials:
* **API Key:** Leave blank or use a dummy value (gateway uses OCI auth).
* **Base URL:** `http://host.docker.internal:8088/v1` (for Podman/Docker; use `http://localhost:8088/v1` if running natively).
* **Base URL**: `http://host.containers.internal:8088/v1` (preferred for Podman on Windows/macOS; fallback to `http://localhost:8088/v1` if running natively or use the gateway's explicit IP with `--add-host host.containers.internal:<gateway-ip>` in Podman run command).
* **Model:** Select or enter an OCI model (list available models via gateway docs or OCI Console).
n8n will now route requests through the gateway to OCI GenAI.

Expand Down Expand Up @@ -126,4 +127,4 @@ This example creates a workflow that triggers on a webhook (e.g., incoming email
## Troubleshooting & Tips
* **Authentication Errors:** Verify `~/.oci/config` permissions and OCI policies (from prerequisites). Test with `oci` CLI commands like `oci os ns get`.
* **Connection Issues:** Ensure port 8088 is open (firewall/OCI security lists). For Podman, use `--network=host` if needed.
* **Model Not Found:** List OCI models in Console under **Analytics & AI > Generative AI**. Ensure your tenancy has the required permissions for `generative-ai-family`.
* **Model Not Found:** List OCI models in Console under **Analytics & AI > Generative AI**. Ensure your tenancy has the required permissions for `generative-ai-family`.
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
3. **[AI-enable](https://docs.oracle.com/en/database/oracle/oracle-database/26/aienb/ai-enablement-guide.pdf) the 26ai database schema.**
4. Install packages:
```bash
pip install oci python-oracledb
pip install oci oracledb
```

## Working Example
Expand Down Expand Up @@ -45,22 +45,17 @@ This is the “query vector” used for vector search in the DB.

```python
import oci
from oci.generative_ai_inference.models import EmbedTextDetails, OnDemandServingMode, CohereEmbedTextRequest

from oci.generative_ai_inference.models import EmbedTextDetails, OnDemandServingMode
def embed_query(genai_client, question: str) -> list[float]:
embed_details = EmbedTextDetails(
compartment_id=os.environ["OCI_COMPARTMENT_ID"],
serving_mode=OnDemandServingMode(model_id=os.environ["EMBED_MODEL_ID"]),
embed_text_request=CohereEmbedTextRequest(
texts=[question],
# Common Cohere v3 pattern:
input_type="SEARCH_QUERY",
),
inputs=[question],
input_type="SEARCH_QUERY",
)
resp = genai_client.embed_text(embed_details)
# Response parsing can differ slightly across SDK versions; this is the common shape.
payload = oci.util.to_dict(resp.data)
vector = payload["embeddings"][0]
# resp.data is EmbedTextResult with .embeddings attribute
vector = resp.data.embeddings[0]
return vector
```

Expand Down Expand Up @@ -143,4 +138,4 @@ def chat_answer(genai_client, prompt: str) -> str:
**What changes depending on needs**

* The request object differs if you switch model families/providers (some use a messages array vs a single `message`).
* generation parameters: `temperature`, `max_tokens`, etc.
* generation parameters: `temperature`, `max_tokens`, etc.