Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs-site/astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ export default defineConfig({
{ label: 'Metadata Filtering', slug: 'lexical-graph/metadata-filtering' },
{ label: 'Reader Providers', slug: 'lexical-graph/readers' },
{ label: 'External Properties', slug: 'lexical-graph/external-properties' },
{ label: 'Ontology-Guided Extraction', slug: 'lexical-graph/ontology-guided-extraction' },
],
},
{
Expand Down
47 changes: 46 additions & 1 deletion docs-site/src/content/docs/lexical-graph/graph-model.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ title: Graph Model
- [Units of context](#units-of-context)
- [Lineage tier](#lineage-tier)
- [Entity-Relationship tier](#entity-relationship-tier)
- [Typed properties on entity nodes](#typed-properties-on-entity-nodes)
- [Summarisation tier](#summarisation-tier)
- [Facts](#facts)
- [Statements](#statements)
Expand Down Expand Up @@ -57,6 +58,50 @@ Extraction uses a lightly guided strategy whereby the extraction process is seed

Relationship values are currently unguided (though relatively concise).

#### Typed properties on entity nodes

Every value extracted from a source is a string. An `__Entity__` node therefore carries a
string `value` (plus a `search_str` for exact-match lookup) and a string `class`, and
nothing on it is numerically comparable:

```
(:__Entity__ {entityId: '…', value: 'Halcyon Motors', class: 'Company',
search_str: 'halcyon motors'})
```

[Ontology-guided extraction](/graphrag-toolkit/lexical-graph/ontology-guided-extraction/)
can add *typed* properties to this tier. When an extracted attribute's predicate resolves
to a datatype property the ontology declares, the value is coerced to the declared XSD type
and written as a native graph property. The write is **additive** – `value`, `search_str`
and `class` are untouched – and where it lands depends on the configured placement:

- **`typed_properties='subject'`** adds the value to the *subject* entity, keyed by the
ontology property's local name. This is what makes an attribute range-queryable from
the entity that has it:

```
(:__Entity__ {entityId: '…', value: 'Halcyon Motors', class: 'Company',
search_str: 'halcyon motors', foundedYear: 1971})
```

- **`typed_properties='complement'`** adds `typed_value` and `datatype` to the
*complement* entity – the local-context node created for the value string itself, which
exists only when `include_local_entities` is enabled:

```
(:__Entity__ {entityId: '…', value: '1971', class: '__Local_Entity__',
typed_value: 1971,
datatype: 'http://www.w3.org/2001/XMLSchema#integer'})
```

These properties describe *the value string*, not the assertion: they say that this
literal read as an integer is 1971, and not which entity was founded then, nor under
which property. The subject and predicate remain on the `__Fact__`.

`typed_value` and `datatype` are reserved by the graph model at complement placement, as
`value`, `search_str` and `class` are at subject placement; an ontology declaring a
property with one of those names is refused at configuration time.

### Summarisation tier

This currently comprises `__Topic__`, `__Statement__` and `__Fact__` nodes. Proceeding from the bottom up:
Expand Down Expand Up @@ -145,7 +190,7 @@ What each node type embeds:

In other words, document-level metadata such as `file_name` or `size` is **not** baked
into chunk embeddings by default — it is carried as graph/source metadata (see
[External properties](/lexical-graph/external-properties/) for making such metadata
[External properties](/graphrag-toolkit/lexical-graph/external-properties/) for making such metadata
queryable). If you want a piece of metadata to influence the embedding itself, include it
in the node's content rather than relying on metadata, since metadata under `source`,
`chunk`, `statement`, and the internal index keys is excluded from the vector.
3 changes: 3 additions & 0 deletions docs-site/src/content/docs/lexical-graph/indexing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ The list of `DEFAULT_ENTITY_CLASSIFICATIONS` used to seed the extraction process

Relationship values are currently unguided (though relatively concise).

If you want a stronger guarantee than "reduces but doesn't eliminate", supply an OWL/RDFS ontology as the extraction vocabulary. The ontology's classes and properties are rendered into the extraction prompt, and a deterministic filter then resolves each extracted name against the ontology – rewriting it to the declared spelling, or, at the strictest level, dropping what doesn't conform. Declared datatype properties can additionally be coerced to their declared type and written as native, queryable graph properties. See [Ontology-Guided Extraction](/graphrag-toolkit/lexical-graph/ontology-guided-extraction/).

#### Build

In the build stage, the LlamaIndex chunk nodes emitted from the extract stage are broken down further into a stream of individual source, chunk, topic, statement and fact LlamaIndex nodes. Graph construction and vector indexing handlers process these nodes to build and index the graph content. Each of these nodes has an `aws::graph::index` metadata item containing data that can be used to index the node in a vector store (though only the chunk and statement nodes are actually indexed in the current implementation).
Expand Down Expand Up @@ -275,6 +277,7 @@ The `ExtractionConfig` object has the following parameters:
| `enable_proposition_extraction` | Perform proposition extraction before extracting topics, statements, facts and entities | `True` |
| `preferred_entity_classifications` | Comma-separated list of preferred entity classifications used to seed the entity extraction | `DEFAULT_ENTITY_CLASSIFICATIONS` |
| `preferred_topics` | List of preferred topic names (or a callable that returns them) supplied to the LLM to seed topic extraction. Accepts the same type as `preferred_entity_classifications`. | `[]` |
| `ontology` | An OWL/RDFS ontology used as the extraction vocabulary. Accepts an `OntologyConfig`, an `Ontology`, an `rdflib.Graph`, or a path to a Turtle (`.ttl`) file – a bare path or graph is equivalent to `OntologyConfig(source)`, which defaults to `ontology_authority='align'` and `typed_properties='off'`. See [Ontology-Guided Extraction](/graphrag-toolkit/lexical-graph/ontology-guided-extraction/). | `None` |
| `infer_entity_classifications` | Determines whether to pre-process documents to identify significant domain entity classifications. Supply either `True` or `False`, or an `InferClassificationsConfig` object. When `True`, an `InferClassifications` step runs as a **pre-processor** before the main extraction loop — one extra LLM round-trip per batch, not per document. | `False` |
| `extract_propositions_prompt_template` | Prompt used to extract propositions from chunks. If `None`, the [default extract propositions template](https://github.com/awslabs/graphrag-toolkit/blob/main/lexical-graph/src/graphrag_toolkit/lexical_graph/indexing/prompts.py#L29-L72) is used. See [Custom prompts](#custom-prompts) below. | `None` |
| `extract_topics_prompt_template` | Prompt used to extract topics, statements and entities from chunks. If `None`, the [default extract topics template](https://github.com/awslabs/graphrag-toolkit/blob/main/lexical-graph/src/graphrag_toolkit/lexical_graph/indexing/prompts.py#L74-L191) is used. See [Custom prompts](#custom-prompts) below. | `None` |
Expand Down
Loading