From ada029d500d5301e2dfc08d64a88df80df8f635f Mon Sep 17 00:00:00 2001 From: Z User Date: Wed, 28 Oct 2026 19:14:01 +0000 Subject: [PATCH] f797e8cd-c1b7-4c35-9432-11571c257b97 --- .../bash_1793214672290_cf47e70d064d.txt | 1206 +++++++++++++++++ .../read_1793214678091_6360aa32baf6.txt | 1206 +++++++++++++++++ .../read_1793214682297_0713917be83f.txt | 517 +++++++ 3 files changed, 2929 insertions(+) create mode 100644 tool-results/bash_1793214672290_cf47e70d064d.txt create mode 100644 tool-results/read_1793214678091_6360aa32baf6.txt create mode 100644 tool-results/read_1793214682297_0713917be83f.txt diff --git a/tool-results/bash_1793214672290_cf47e70d064d.txt b/tool-results/bash_1793214672290_cf47e70d064d.txt new file mode 100644 index 0000000..880c508 --- /dev/null +++ b/tool-results/bash_1793214672290_cf47e70d064d.txt @@ -0,0 +1,1206 @@ +--- +name: graphify +description: any input (code, docs, papers, images) → knowledge graph → clustered communities → HTML + JSON + audit report +trigger: /graphify +--- + +# /graphify + +Turn any folder of files into a navigable knowledge graph with community detection, an honest audit trail, and three outputs: interactive HTML, GraphRAG-ready JSON, and a plain-language GRAPH_REPORT.md. + +## Usage + +``` +/graphify # full pipeline on current directory → Obsidian vault +/graphify # full pipeline on specific path +/graphify --mode deep # thorough extraction, richer INFERRED edges +/graphify --update # incremental - re-extract only new/changed files +/graphify --cluster-only # rerun clustering on existing graph +/graphify --no-viz # skip visualization, just report + JSON +/graphify --html # (HTML is generated by default - this flag is a no-op) +/graphify --svg # also export graph.svg (embeds in Notion, GitHub) +/graphify --graphml # export graph.graphml (Gephi, yEd) +/graphify --neo4j # generate graphify-out/cypher.txt for Neo4j +/graphify --neo4j-push bolt://localhost:7687 # push directly to Neo4j +/graphify --mcp # start MCP stdio server for agent access +/graphify --watch # watch folder, auto-rebuild on code changes (no LLM needed) +/graphify add # fetch URL, save to ./raw, update graph +/graphify add --author "Name" # tag who wrote it +/graphify add --contributor "Name" # tag who added it to the corpus +/graphify query "" # BFS traversal - broad context +/graphify query "" --dfs # DFS - trace a specific path +/graphify query "" --budget 1500 # cap answer at N tokens +/graphify path "AuthModule" "Database" # shortest path between two concepts +/graphify explain "SwinTransformer" # plain-language explanation of a node +``` + +## What graphify is for + +graphify is built around Andrej Karpathy's /raw folder workflow: drop anything into a folder - papers, tweets, screenshots, code, notes - and get a structured knowledge graph that shows you what you didn't know was connected. + +Three things it does that Claude alone cannot: +1. **Persistent graph** - relationships are stored in `graphify-out/graph.json` and survive across sessions. Ask questions weeks later without re-reading everything. +2. **Honest audit trail** - every edge is tagged EXTRACTED, INFERRED, or AMBIGUOUS. You know what was found vs invented. +3. **Cross-document surprise** - community detection finds connections between concepts in different files that you would never think to ask about directly. + +Use it for: +- A codebase you're new to (understand architecture before touching anything) +- A reading list (papers + tweets + notes → one navigable graph) +- A research corpus (citation graph + concept graph in one) +- Your personal /raw folder (drop everything in, let it grow, query it) + +## What You Must Do When Invoked + +If no path was given, use `.` (current directory). Do not ask the user for a path. + +Follow these steps in order. Do not skip steps. + +### Step 1 - Ensure graphify is installed + +```bash +python3 -c "import graphify" 2>/dev/null || pip install graphifyy -q --break-system-packages 2>&1 | tail -3 +``` + +If the import succeeds, print nothing and move straight to Step 2. + +### Step 2 - Detect files + +```bash +python3 -c " +import json +from graphify.detect import detect +from pathlib import Path +result = detect(Path('INPUT_PATH')) +print(json.dumps(result)) +" > .graphify_detect.json +``` + +Replace INPUT_PATH with the actual path the user provided. Do NOT cat or print the JSON - read it silently and present a clean summary instead: + +``` +Corpus: X files · ~Y words + code: N files (.py .ts .go ...) + docs: N files (.md .txt ...) + papers: N files (.pdf ...) + images: N files +``` + +Then act on it: +- If `total_files` is 0: stop with "No supported files found in [path]." +- If `skipped_sensitive` is non-empty: mention file count skipped, not the file names. +- If `total_words` > 2,000,000 OR `total_files` > 200: show the warning and the top 5 subdirectories by file count, then ask which subfolder to run on. Wait for the user's answer before proceeding. +- Otherwise: proceed directly to Step 3 - no need to ask anything. + +### Step 3 - Extract entities and relationships + +**Before starting:** note whether `--mode deep` was given. You must pass `DEEP_MODE=true` to every subagent in Step B2 if it was. Track this from the original invocation - do not lose it. + +This step has two parts: **structural extraction** (deterministic, free) and **semantic extraction** (Claude, costs tokens). + +**Run Part A (AST) and Part B (semantic) in parallel. Dispatch all semantic subagents AND start AST extraction in the same message. Both can run simultaneously since they operate on different file types. Merge results in Part C as before.** + +Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is deterministic and fast; start it while subagents are processing docs/papers. + +#### Part A - Structural extraction for code files + +For any code files detected, run AST extraction in parallel with Part B subagents: + +```bash +python3 -c " +import sys, json +from graphify.extract import collect_files, extract +from pathlib import Path +import json + +code_files = [] +detect = json.loads(Path('.graphify_detect.json').read_text()) +for f in detect.get('files', {}).get('code', []): + code_files.extend(collect_files(Path(f)) if Path(f).is_dir() else [Path(f)]) + +if code_files: + result = extract(code_files) + Path('.graphify_ast.json').write_text(json.dumps(result, indent=2)) + print(f'AST: {len(result[\"nodes\"])} nodes, {len(result[\"edges\"])} edges') +else: + Path('.graphify_ast.json').write_text(json.dumps({'nodes':[],'edges':[],'input_tokens':0,'output_tokens':0})) + print('No code files - skipping AST extraction') +" +``` + +#### Part B - Semantic extraction (parallel subagents) + +**Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. + +**MANDATORY: You MUST use the Agent tool here. Reading files yourself one-by-one is forbidden - it is 5-10x slower. If you do not use the Agent tool you are doing this wrong.** + +Before dispatching subagents, print a timing estimate: +- Load `total_words` and file counts from `.graphify_detect.json` +- Estimate agents needed: `ceil(uncached_non_code_files / 22)` (chunk size is 20-25) +- Estimate time: ~45s per agent batch (they run in parallel, so total ≈ 45s × ceil(agents/parallel_limit)) +- Print: "Semantic extraction: ~N files → X agents, estimated ~Ys" + +**Step B0 - Check extraction cache first** + +Before dispatching any subagents, check which files already have cached extraction results: + +```bash +python3 -c " +import json +from graphify.cache import check_semantic_cache +from pathlib import Path + +detect = json.loads(Path('.graphify_detect.json').read_text()) +all_files = [f for files in detect['files'].values() for f in files] + +cached_nodes, cached_edges, uncached = check_semantic_cache(all_files) + +if cached_nodes or cached_edges: + Path('.graphify_cached.json').write_text(json.dumps({'nodes': cached_nodes, 'edges': cached_edges})) +Path('.graphify_uncached.txt').write_text('\n'.join(uncached)) +print(f'Cache: {len(all_files)-len(uncached)} files hit, {len(uncached)} files need extraction') +" +``` + +Only dispatch subagents for files listed in `.graphify_uncached.txt`. If all files are cached, skip to Part C directly. + +**Step B1 - Split into chunks** + +Load files from `.graphify_uncached.txt`. Split into chunks of 20-25 files each. Each image gets its own chunk (vision needs separate context). + +**Step B2 - Dispatch ALL subagents in a single message** + +Call the Agent tool multiple times IN THE SAME RESPONSE - one call per chunk. This is the only way they run in parallel. If you make one Agent call, wait, then make another, you are doing it sequentially and defeating the purpose. + +Concrete example for 3 chunks: +``` +[Agent tool call 1: files 1-15] +[Agent tool call 2: files 16-30] +[Agent tool call 3: files 31-45] +``` +All three in one message. Not three separate messages. + +Each subagent receives this exact prompt (substitute FILE_LIST, CHUNK_NUM, TOTAL_CHUNKS, and DEEP_MODE): + +``` +You are a graphify extraction subagent. Read the files listed and extract a knowledge graph fragment. +Output ONLY valid JSON matching the schema below - no explanation, no markdown fences, no preamble. + +Files (chunk CHUNK_NUM of TOTAL_CHUNKS): +FILE_LIST + +Rules: +- EXTRACTED: relationship explicit in source (import, call, citation, "see §3.2") +- INFERRED: reasonable inference (shared data structure, implied dependency) +- AMBIGUOUS: uncertain - flag for review, do not omit + +Code files: focus on semantic edges AST cannot find (call relationships, shared data, arch patterns). + Do not re-extract imports - AST already has those. +Doc/paper files: extract named concepts, entities, citations. +Image files: use vision to understand what the image IS - do not just OCR. + UI screenshot: layout patterns, design decisions, key elements, purpose. + Chart: metric, trend/insight, data source. + Tweet/post: claim as node, author, concepts mentioned. + Diagram: components and connections. + Research figure: what it demonstrates, method, result. + Handwritten/whiteboard: ideas and arrows, mark uncertain readings AMBIGUOUS. + +DEEP_MODE (if --mode deep was given): be aggressive with INFERRED edges - indirect deps, + shared assumptions, latent couplings. Mark uncertain ones AMBIGUOUS instead of omitting. + +Semantic similarity: if two concepts in this chunk solve the same problem or represent the same idea without any structural link (no import, no call, no citation), add a `semantically_similar_to` edge marked INFERRED with a confidence_score reflecting how similar they are (0.6-0.95). Examples: +- Two functions that both validate user input but never call each other +- A class in code and a concept in a paper that describe the same algorithm +- Two error types that handle the same failure mode differently +Only add these when the similarity is genuinely non-obvious and cross-cutting. Do not add them for trivially similar things. + +Hyperedges: if 3 or more nodes clearly participate together in a shared concept, flow, or pattern that is not captured by pairwise edges alone, add a hyperedge to a top-level `hyperedges` array. Examples: +- All classes that implement a common protocol or interface +- All functions in an authentication flow (even if they don't all call each other) +- All concepts from a paper section that form one coherent idea +Use sparingly — only when the group relationship adds information beyond the pairwise edges. Maximum 3 hyperedges per chunk. + +If a file has YAML frontmatter (--- ... ---), copy source_url, captured_at, author, + contributor onto every node from that file. + +confidence_score rules: +- EXTRACTED edges: confidence_score must be 1.0 +- INFERRED edges: score 0.4-0.9 based on how certain you are. + Strong structural inference (e.g. two classes clearly share data): 0.8-0.9. + Reasonable but not certain: 0.6-0.7. Weak inference: 0.4-0.5. +- AMBIGUOUS edges: score 0.1-0.3 + +Output exactly this JSON (no other text): +{"nodes":[{"id":"filestem_entityname","label":"Human Readable Name","file_type":"code|document|paper|image","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0} +``` + +**Step B3 - Collect, cache, and merge** + +Wait for all subagents. For each result: +- If a subagent returned valid JSON with `nodes` and `edges`, include it and save each file's nodes/edges to the cache +- If a subagent failed or returned invalid JSON, print a warning and skip that chunk - do not abort + +If more than half the chunks failed, stop and tell the user. + +Save new results to cache: +```bash +python3 -c " +import json +from graphify.cache import save_semantic_cache +from pathlib import Path + +new = json.loads(Path('.graphify_semantic_new.json').read_text()) if Path('.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[]} +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', [])) +print(f'Cached {saved} files') +" +``` + +Merge cached + new results into `.graphify_semantic.json`: +```bash +python3 -c " +import json +from pathlib import Path + +cached = json.loads(Path('.graphify_cached.json').read_text()) if Path('.graphify_cached.json').exists() else {'nodes':[],'edges':[]} +new = json.loads(Path('.graphify_semantic_new.json').read_text()) if Path('.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[]} + +all_nodes = cached['nodes'] + new.get('nodes', []) +all_edges = cached['edges'] + new.get('edges', []) +seen = set() +deduped = [] +for n in all_nodes: + if n['id'] not in seen: + seen.add(n['id']) + deduped.append(n) + +merged = { + 'nodes': deduped, + 'edges': all_edges, + 'input_tokens': new.get('input_tokens', 0), + 'output_tokens': new.get('output_tokens', 0), +} +Path('.graphify_semantic.json').write_text(json.dumps(merged, indent=2)) +print(f'Extraction complete - {len(deduped)} nodes, {len(all_edges)} edges ({len(cached[\"nodes\"])} from cache, {len(new.get(\"nodes\",[]))} new)') +" +``` +Clean up temp files: `rm -f .graphify_cached.json .graphify_uncached.txt .graphify_semantic_new.json` + +#### Part C - Merge AST + semantic into final extraction + +```bash +python3 -c " +import sys, json +from pathlib import Path + +ast = json.loads(Path('.graphify_ast.json').read_text()) +sem = json.loads(Path('.graphify_semantic.json').read_text()) + +# Merge: AST nodes first, semantic nodes deduplicated by id +seen = {n['id'] for n in ast['nodes']} +merged_nodes = list(ast['nodes']) +for n in sem['nodes']: + if n['id'] not in seen: + merged_nodes.append(n) + seen.add(n['id']) + +merged_edges = ast['edges'] + sem['edges'] +merged = { + 'nodes': merged_nodes, + 'edges': merged_edges, + 'input_tokens': sem.get('input_tokens', 0), + 'output_tokens': sem.get('output_tokens', 0), +} +Path('.graphify_extract.json').write_text(json.dumps(merged, indent=2)) +total = len(merged_nodes) +edges = len(merged_edges) +print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(sem[\"nodes\"])} semantic)') +" +``` + +### Step 4 - Build graph, cluster, analyze, generate outputs + +```bash +mkdir -p graphify-out +python3 -c " +import sys, json +from graphify.build import build_from_json +from graphify.cluster import cluster, score_all +from graphify.analyze import god_nodes, surprising_connections, suggest_questions +from graphify.report import generate +from graphify.export import to_json +from pathlib import Path + +extraction = json.loads(Path('.graphify_extract.json').read_text()) +detection = json.loads(Path('.graphify_detect.json').read_text()) + +G = build_from_json(extraction) +communities = cluster(G) +cohesion = score_all(G, communities) +tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} +gods = god_nodes(G) +surprises = surprising_connections(G, communities) +labels = {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 +questions = suggest_questions(G, communities, labels) + +report = generate(G, communities, cohesion, labels, gods, surprises, detection, tokens, 'INPUT_PATH', suggested_questions=questions) +Path('graphify-out/GRAPH_REPORT.md').write_text(report) +to_json(G, communities, 'graphify-out/graph.json') + +analysis = { + 'communities': {str(k): v for k, v in communities.items()}, + 'cohesion': {str(k): v for k, v in cohesion.items()}, + 'gods': gods, + 'surprises': surprises, + 'questions': questions, +} +Path('.graphify_analysis.json').write_text(json.dumps(analysis, indent=2)) +if G.number_of_nodes() == 0: + print('ERROR: Graph is empty - extraction produced no nodes.') + print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') + raise SystemExit(1) +print(f'Graph: {G.number_of_nodes()} nodes, {G.number_of_edges()} edges, {len(communities)} communities') +" +``` + +If this step prints `ERROR: Graph is empty`, stop and tell the user what happened - do not proceed to labeling or visualization. + +Replace INPUT_PATH with the actual path. + +### Step 5 - Label communities + +Read `.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). + +Then regenerate the report and save the labels for the visualizer: + +```bash +python3 -c " +import sys, json +from graphify.build import build_from_json +from graphify.cluster import score_all +from graphify.analyze import god_nodes, surprising_connections, suggest_questions +from graphify.report import generate +from pathlib import Path + +extraction = json.loads(Path('.graphify_extract.json').read_text()) +detection = json.loads(Path('.graphify_detect.json').read_text()) +analysis = json.loads(Path('.graphify_analysis.json').read_text()) + +G = build_from_json(extraction) +communities = {int(k): v for k, v in analysis['communities'].items()} +cohesion = {int(k): v for k, v in analysis['cohesion'].items()} +tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} + +# LABELS - replace these with the names you chose above +labels = LABELS_DICT + +# Regenerate questions with real community labels (labels affect question phrasing) +questions = suggest_questions(G, communities, labels) + +report = generate(G, communities, cohesion, labels, analysis['gods'], analysis['surprises'], detection, tokens, 'INPUT_PATH', suggested_questions=questions) +Path('graphify-out/GRAPH_REPORT.md').write_text(report) +Path('.graphify_labels.json').write_text(json.dumps({str(k): v for k, v in labels.items()})) +print('Report updated with community labels') +" +``` + +Replace `LABELS_DICT` with the actual dict you constructed (e.g. `{0: "Attention Mechanism", 1: "Training Pipeline"}`). +Replace INPUT_PATH with the actual path. + +### Step 6 - Generate Obsidian vault (default) + optional HTML + +**Always generate the Obsidian vault and HTML** - they are the primary visualizations. Skip both if `--no-viz` (report + JSON only). + +```bash +python3 -c " +import sys, json +from graphify.build import build_from_json +from graphify.export import to_obsidian, to_canvas +from pathlib import Path + +extraction = json.loads(Path('.graphify_extract.json').read_text()) +analysis = json.loads(Path('.graphify_analysis.json').read_text()) +labels_raw = json.loads(Path('.graphify_labels.json').read_text()) if Path('.graphify_labels.json').exists() else {} + +G = build_from_json(extraction) +communities = {int(k): v for k, v in analysis['communities'].items()} +cohesion = {int(k): v for k, v in analysis['cohesion'].items()} +labels = {int(k): v for k, v in labels_raw.items()} + +n = to_obsidian(G, communities, 'graphify-out/obsidian', community_labels=labels or None, cohesion=cohesion) +print(f'Obsidian vault: {n} notes in graphify-out/obsidian/') + +to_canvas(G, communities, 'graphify-out/obsidian/graph.canvas', community_labels=labels or None) +print('Canvas: graphify-out/obsidian/graph.canvas - open in Obsidian for structured community layout') +print() +print('Open graphify-out/obsidian/ as a vault in Obsidian.') +print(' Graph view - nodes colored by community (set automatically)') +print(' graph.canvas - structured layout with communities as groups') +print(' _COMMUNITY_* - overview notes with cohesion scores and dataview queries') +" +``` + +Also generate the HTML graph (always, unless `--no-viz`): + +```bash +python3 -c " +import sys, json +from graphify.build import build_from_json +from graphify.export import to_html +from pathlib import Path + +extraction = json.loads(Path('.graphify_extract.json').read_text()) +analysis = json.loads(Path('.graphify_analysis.json').read_text()) +labels_raw = json.loads(Path('.graphify_labels.json').read_text()) if Path('.graphify_labels.json').exists() else {} + +G = build_from_json(extraction) +communities = {int(k): v for k, v in analysis['communities'].items()} +labels = {int(k): v for k, v in labels_raw.items()} + +if G.number_of_nodes() > 5000: + print(f'Graph has {G.number_of_nodes()} nodes - too large for HTML viz. Use Obsidian vault instead.') +else: + to_html(G, communities, 'graphify-out/graph.html', community_labels=labels or None) + print('graph.html written - open in any browser, no server needed') +" +``` + +### Step 7 - Neo4j export (only if --neo4j or --neo4j-push flag) + +**If `--neo4j`** - generate a Cypher file for manual import: + +```bash +python3 -c " +import sys, json +from graphify.build import build_from_json +from graphify.export import to_cypher +from pathlib import Path + +G = build_from_json(json.loads(Path('.graphify_extract.json').read_text())) +to_cypher(G, 'graphify-out/cypher.txt') +print('cypher.txt written - import with: cypher-shell < graphify-out/cypher.txt') +" +``` + +**If `--neo4j-push `** - push directly to a running Neo4j instance. Ask the user for credentials if not provided: + +```bash +python3 -c " +import sys, json +from graphify.build import build_from_json +from graphify.cluster import cluster +from graphify.export import push_to_neo4j +from pathlib import Path + +extraction = json.loads(Path('.graphify_extract.json').read_text()) +analysis = json.loads(Path('.graphify_analysis.json').read_text()) +G = build_from_json(extraction) +communities = {int(k): v for k, v in analysis['communities'].items()} + +result = push_to_neo4j(G, uri='NEO4J_URI', user='NEO4J_USER', password='NEO4J_PASSWORD', communities=communities) +print(f'Pushed to Neo4j: {result[\"nodes\"]} nodes, {result[\"edges\"]} edges') +" +``` + +Replace `NEO4J_URI`, `NEO4J_USER`, `NEO4J_PASSWORD` with actual values. Default URI is `bolt://localhost:7687`, default user is `neo4j`. Uses MERGE - safe to re-run without creating duplicates. + +### Step 7b - SVG export (only if --svg flag) + +```bash +python3 -c " +import sys, json +from graphify.build import build_from_json +from graphify.export import to_svg +from pathlib import Path + +extraction = json.loads(Path('.graphify_extract.json').read_text()) +analysis = json.loads(Path('.graphify_analysis.json').read_text()) +labels_raw = json.loads(Path('.graphify_labels.json').read_text()) if Path('.graphify_labels.json').exists() else {} + +G = build_from_json(extraction) +communities = {int(k): v for k, v in analysis['communities'].items()} +labels = {int(k): v for k, v in labels_raw.items()} + +to_svg(G, communities, 'graphify-out/graph.svg', community_labels=labels or None) +print('graph.svg written - embeds in Obsidian, Notion, GitHub READMEs') +" +``` + +### Step 7c - GraphML export (only if --graphml flag) + +```bash +python3 -c " +import json +from graphify.build import build_from_json +from graphify.export import to_graphml +from pathlib import Path + +extraction = json.loads(Path('.graphify_extract.json').read_text()) +analysis = json.loads(Path('.graphify_analysis.json').read_text()) + +G = build_from_json(extraction) +communities = {int(k): v for k, v in analysis['communities'].items()} + +to_graphml(G, communities, 'graphify-out/graph.graphml') +print('graph.graphml written - open in Gephi, yEd, or any GraphML tool') +" +``` + +### Step 7d - MCP server (only if --mcp flag) + +```bash +python3 -m graphify.serve graphify-out/graph.json +``` + +This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. + +To configure in Claude Desktop, add to `claude_desktop_config.json`: +```json +{ + "mcpServers": { + "graphify": { + "command": "python3", + "args": ["-m", "graphify.serve", "/absolute/path/to/graphify-out/graph.json"] + } + } +} +``` + +### Step 8 - Token reduction benchmark (only if total_words > 5000) + +If `total_words` from `.graphify_detect.json` is greater than 5,000, run: + +```bash +python3 -c " +import json +from graphify.benchmark import run_benchmark, print_benchmark +from pathlib import Path + +detection = json.loads(Path('.graphify_detect.json').read_text()) +result = run_benchmark('graphify-out/graph.json', corpus_words=detection['total_words']) +print_benchmark(result) +" +``` + +Print the output directly in chat. If `total_words <= 5000`, skip silently - the graph value is structural clarity, not token compression, for small corpora. + +--- + +### Step 9 - Save manifest, update cost tracker, clean up, and report + +```bash +python3 -c " +import json +from pathlib import Path +from datetime import datetime, timezone +from graphify.detect import save_manifest + +# Save manifest for --update +detect = json.loads(Path('.graphify_detect.json').read_text()) +save_manifest(detect['files']) + +# Update cumulative cost tracker +extract = json.loads(Path('.graphify_extract.json').read_text()) +input_tok = extract.get('input_tokens', 0) +output_tok = extract.get('output_tokens', 0) + +cost_path = Path('graphify-out/cost.json') +if cost_path.exists(): + cost = json.loads(cost_path.read_text()) +else: + cost = {'runs': [], 'total_input_tokens': 0, 'total_output_tokens': 0} + +cost['runs'].append({ + 'date': datetime.now(timezone.utc).isoformat(), + 'input_tokens': input_tok, + 'output_tokens': output_tok, + 'files': detect.get('total_files', 0), +}) +cost['total_input_tokens'] += input_tok +cost['total_output_tokens'] += output_tok +cost_path.write_text(json.dumps(cost, indent=2)) + +print(f'This run: {input_tok:,} input tokens, {output_tok:,} output tokens') +print(f'All time: {cost[\"total_input_tokens\"]:,} input, {cost[\"total_output_tokens\"]:,} output ({len(cost[\"runs\"])} runs)') +" +rm -f .graphify_detect.json .graphify_extract.json .graphify_ast.json .graphify_semantic.json .graphify_analysis.json .graphify_labels.json +rm -f graphify-out/.needs_update 2>/dev/null || true +``` + +Tell the user: +``` +Graph complete. Outputs are in a hidden folder called graphify-out/ inside the directory you ran this on. + +The folder is hidden (dot prefix) so it won't show in Finder or a normal ls. +To see it: + Mac/Linux: ls -la graphify-out/ + VS Code: the Explorer panel shows hidden files by default + Finder: Cmd+Shift+. to toggle hidden files + +What's inside: + graphify-out/obsidian/ - open this folder as a vault in Obsidian (File > Open Vault) + graphify-out/GRAPH_REPORT.md - full audit report, also readable here in Claude + graphify-out/graph.json - persistent graph, query it later with /graphify query "..." + +Full path: PATH_TO_DIR/graphify-out/ +``` + +Replace PATH_TO_DIR with the actual absolute path of the directory that was processed. + +Then paste these sections from GRAPH_REPORT.md directly into the chat: +- God Nodes +- Surprising Connections +- Suggested Questions + +Do NOT paste the full report - just those three sections. Keep it concise. + +Then immediately offer to explore. Pick the single most interesting suggested question from the report - the one that crosses the most community boundaries or has the most surprising bridge node - and ask: + +> "The most interesting question this graph can answer: **[question]**. Want me to trace it?" + +If the user says yes, run `/graphify query "[question]"` on the graph and walk them through the answer using the graph structure - which nodes connect, which community boundaries get crossed, what the path reveals. Keep going as long as they want to explore. Each answer should end with a natural follow-up ("this connects to X - want to go deeper?") so the session feels like navigation, not a one-shot report. + +The graph is the map. Your job after the pipeline is to be the guide. + +--- + +## For --update (incremental re-extraction) + +Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. + +```bash +python3 -c " +import sys, json +from graphify.detect import detect_incremental, save_manifest +from pathlib import Path + +result = detect_incremental(Path('INPUT_PATH')) +new_total = result.get('new_total', 0) +print(json.dumps(result, indent=2)) +Path('.graphify_incremental.json').write_text(json.dumps(result)) +if new_total == 0: + print('No files changed since last run. Nothing to update.') + raise SystemExit(0) +print(f'{new_total} new/changed file(s) to re-extract.') +" +``` + +If new files exist, first check whether all changed files are code files: + +```bash +python3 -c " +import json +from pathlib import Path + +result = json.loads(open('.graphify_incremental.json').read()) if Path('.graphify_incremental.json').exists() else {} +code_exts = {'.py','.ts','.js','.go','.rs','.java','.cpp','.c','.rb','.swift','.kt','.cs','.scala','.php','.cc','.cxx','.hpp','.h','.kts'} +new_files = result.get('new_files', {}) +all_changed = [f for files in new_files.values() for f in files] +code_only = all(Path(f).suffix.lower() in code_exts for f in all_changed) +print('code_only:', code_only) +" +``` + +If `code_only` is True: print `[graphify update] Code-only changes detected - skipping semantic extraction (no LLM needed)`, run only Step 3A (AST) on the changed files, skip Step 3B entirely (no subagents), then go straight to merge and Steps 4–8. + +If `code_only` is False (any changed file is a doc/paper/image): run the full Steps 3A–3C pipeline as normal. + +Then: + +```bash +python3 -c " +import sys, json +from graphify.build import build_from_json +from graphify.export import to_json +from networkx.readwrite import json_graph +import networkx as nx +from pathlib import Path + +# Load existing graph +existing_data = json.loads(Path('graphify-out/graph.json').read_text()) +G_existing = json_graph.node_link_graph(existing_data, edges='links') + +# Load new extraction +new_extraction = json.loads(Path('.graphify_extract.json').read_text()) +G_new = build_from_json(new_extraction) + +# Merge: new nodes/edges into existing graph +G_existing.update(G_new) +print(f'Merged: {G_existing.number_of_nodes()} nodes, {G_existing.number_of_edges()} edges') +" +``` + +Then run Steps 4–8 on the merged graph as normal. + +After Step 4, show the graph diff: + +```bash +python3 -c " +import json +from graphify.analyze import graph_diff +from graphify.build import build_from_json +from networkx.readwrite import json_graph +import networkx as nx +from pathlib import Path + +# Load old graph (before update) from backup written before merge +old_data = json.loads(Path('.graphify_old.json').read_text()) if Path('.graphify_old.json').exists() else None +new_extract = json.loads(Path('.graphify_extract.json').read_text()) +G_new = build_from_json(new_extract) + +if old_data: + G_old = json_graph.node_link_graph(old_data, edges='links') + diff = graph_diff(G_old, G_new) + print(diff['summary']) + if diff['new_nodes']: + print('New nodes:', ', '.join(n['label'] for n in diff['new_nodes'][:5])) + if diff['new_edges']: + print('New edges:', len(diff['new_edges'])) +" +``` + +Before the merge step, save the old graph: `cp graphify-out/graph.json .graphify_old.json` +Clean up after: `rm -f .graphify_old.json` + +--- + +## For --cluster-only + +Skip Steps 1–3. Load the existing graph from `graphify-out/graph.json` and re-run clustering: + +```bash +python3 -c " +import sys, json +from graphify.cluster import cluster, score_all +from graphify.analyze import god_nodes, surprising_connections +from graphify.report import generate +from graphify.export import to_json +from networkx.readwrite import json_graph +import networkx as nx +from pathlib import Path + +data = json.loads(Path('graphify-out/graph.json').read_text()) +G = json_graph.node_link_graph(data, edges='links') + +detection = {'total_files': 0, 'total_words': 99999, 'needs_graph': True, 'warning': None, + 'files': {'code': [], 'document': [], 'paper': []}} +tokens = {'input': 0, 'output': 0} + +communities = cluster(G) +cohesion = score_all(G, communities) +gods = god_nodes(G) +surprises = surprising_connections(G, communities) +labels = {cid: 'Community ' + str(cid) for cid in communities} + +report = generate(G, communities, cohesion, labels, gods, surprises, detection, tokens, '.') +Path('graphify-out/GRAPH_REPORT.md').write_text(report) +to_json(G, communities, 'graphify-out/graph.json') + +analysis = { + 'communities': {str(k): v for k, v in communities.items()}, + 'cohesion': {str(k): v for k, v in cohesion.items()}, + 'gods': gods, + 'surprises': surprises, +} +Path('.graphify_analysis.json').write_text(json.dumps(analysis, indent=2)) +print(f'Re-clustered: {len(communities)} communities') +" +``` + +Then run Steps 5–9 as normal (label communities, generate viz, benchmark, clean up, report). + +--- + +## For /graphify query + +Two traversal modes - choose based on the question: + +| Mode | Flag | Best for | +|------|------|----------| +| BFS (default) | _(none)_ | "What is X connected to?" - broad context, nearest neighbors first | +| DFS | `--dfs` | "How does X reach Y?" - trace a specific chain or dependency path | + +First check the graph exists: +```bash +python3 -c " +from pathlib import Path +if not Path('graphify-out/graph.json').exists(): + print('ERROR: No graph found. Run /graphify first to build the graph.') + raise SystemExit(1) +" +``` +If it fails, stop and tell the user to run `/graphify ` first. + +Load `graphify-out/graph.json`, then: + +1. Find the 1-3 nodes whose label best matches key terms in the question. +2. Run the appropriate traversal from each starting node. +3. Read the subgraph - node labels, edge relations, confidence tags, source locations. +4. Answer using **only** what the graph contains. Quote `source_location` when citing a specific fact. +5. If the graph lacks enough information, say so - do not hallucinate edges. + +```bash +python3 -c " +import sys, json +from networkx.readwrite import json_graph +import networkx as nx +from pathlib import Path + +data = json.loads(Path('graphify-out/graph.json').read_text()) +G = json_graph.node_link_graph(data, edges='links') + +question = 'QUESTION' +mode = 'MODE' # 'bfs' or 'dfs' +terms = [t.lower() for t in question.split() if len(t) > 3] + +# Find best-matching start nodes +scored = [] +for nid, ndata in G.nodes(data=True): + label = ndata.get('label', '').lower() + score = sum(1 for t in terms if t in label) + if score > 0: + scored.append((score, nid)) +scored.sort(reverse=True) +start_nodes = [nid for _, nid in scored[:3]] + +if not start_nodes: + print('No matching nodes found for query terms:', terms) + sys.exit(0) + +subgraph_nodes = set() +subgraph_edges = [] + +if mode == 'dfs': + # DFS: follow one path as deep as possible before backtracking. + # Depth-limited to 6 to avoid traversing the whole graph. + visited = set() + stack = [(n, 0) for n in reversed(start_nodes)] + while stack: + node, depth = stack.pop() + if node in visited or depth > 6: + continue + visited.add(node) + subgraph_nodes.add(node) + for neighbor in G.neighbors(node): + if neighbor not in visited: + stack.append((neighbor, depth + 1)) + subgraph_edges.append((node, neighbor)) +else: + # BFS: explore all neighbors layer by layer up to depth 3. + frontier = set(start_nodes) + subgraph_nodes = set(start_nodes) + for _ in range(3): + next_frontier = set() + for n in frontier: + for neighbor in G.neighbors(n): + if neighbor not in subgraph_nodes: + next_frontier.add(neighbor) + subgraph_edges.append((n, neighbor)) + subgraph_nodes.update(next_frontier) + frontier = next_frontier + +# Token-budget aware output: rank by relevance, cut at budget (~4 chars/token) +token_budget = BUDGET # default 2000 +char_budget = token_budget * 4 + +# Score each node by term overlap for ranked output +def relevance(nid): + label = G.nodes[nid].get('label', '').lower() + return sum(1 for t in terms if t in label) + +ranked_nodes = sorted(subgraph_nodes, key=relevance, reverse=True) + +lines = [f'Traversal: {mode.upper()} | Start: {[G.nodes[n].get(\"label\",n) for n in start_nodes]} | {len(subgraph_nodes)} nodes'] +for nid in ranked_nodes: + d = G.nodes[nid] + lines.append(f' NODE {d.get(\"label\", nid)} [src={d.get(\"source_file\",\"\")} loc={d.get(\"source_location\",\"\")}]') +for u, v in subgraph_edges: + if u in subgraph_nodes and v in subgraph_nodes: + d = G.edges[u, v] + lines.append(f' EDGE {G.nodes[u].get(\"label\",u)} --{d.get(\"relation\",\"\")} [{d.get(\"confidence\",\"\")}]--> {G.nodes[v].get(\"label\",v)}') + +output = '\n'.join(lines) +if len(output) > char_budget: + output = output[:char_budget] + f'\n... (truncated at ~{token_budget} token budget - use --budget N for more)' +print(output) +" +``` + +Replace `QUESTION` with the user's actual question, `MODE` with `bfs` or `dfs`, and `BUDGET` with the token budget (default `2000`, or whatever `--budget N` specifies). Then answer based on the subgraph output above. + +After writing the answer, save it back into the graph so it improves future queries: + +```bash +python3 -c " +from graphify.ingest import save_query_result +from pathlib import Path +save_query_result( + question='QUESTION', + answer='ANSWER', + memory_dir=Path('graphify-out/memory'), + query_type='query', + source_nodes=SOURCE_NODES, # list of node labels cited, or [] +) +print('Query result saved to graphify-out/memory/') +" +``` + +Replace `QUESTION` with the question, `ANSWER` with your full answer text, `SOURCE_NODES` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. + +--- + +## For /graphify path + +Find the shortest path between two named concepts in the graph. + +First check the graph exists: +```bash +python3 -c " +from pathlib import Path +if not Path('graphify-out/graph.json').exists(): + print('ERROR: No graph found. Run /graphify first to build the graph.') + raise SystemExit(1) +" +``` +If it fails, stop and tell the user to run `/graphify ` first. + +```bash +python3 -c " +import json, sys +import networkx as nx +from networkx.readwrite import json_graph +from pathlib import Path + +data = json.loads(Path('graphify-out/graph.json').read_text()) +G = json_graph.node_link_graph(data, edges='links') + +a_term = 'NODE_A' +b_term = 'NODE_B' + +def find_node(term): + term = term.lower() + scored = sorted( + [(sum(1 for w in term.split() if w in G.nodes[n].get('label','').lower()), n) + for n in G.nodes()], + reverse=True + ) + return scored[0][1] if scored and scored[0][0] > 0 else None + +src = find_node(a_term) +tgt = find_node(b_term) + +if not src or not tgt: + print(f'Could not find nodes matching: {a_term!r} or {b_term!r}') + sys.exit(0) + +try: + path = nx.shortest_path(G, src, tgt) + print(f'Shortest path ({len(path)-1} hops):') + for i, nid in enumerate(path): + label = G.nodes[nid].get('label', nid) + if i < len(path) - 1: + edge = G.edges[nid, path[i+1]] + rel = edge.get('relation', '') + conf = edge.get('confidence', '') + print(f' {label} --{rel}--> [{conf}]') + else: + print(f' {label}') +except nx.NetworkXNoPath: + print(f'No path found between {a_term!r} and {b_term!r}') +except nx.NodeNotFound as e: + print(f'Node not found: {e}') +" +``` + +Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then explain the path in plain language - what each hop means, why it's significant. + +After writing the explanation, save it back: + +```bash +python3 -c " +from graphify.ingest import save_query_result +from pathlib import Path +save_query_result( + question='Path from NODE_A to NODE_B', + answer='ANSWER', + memory_dir=Path('graphify-out/memory'), + query_type='path_query', + source_nodes=PATH_NODES, # list of node labels on the path +) +print('Path result saved to graphify-out/memory/') +" +``` + +--- + +## For /graphify explain + +Give a plain-language explanation of a single node - everything connected to it. + +First check the graph exists: +```bash +python3 -c " +from pathlib import Path +if not Path('graphify-out/graph.json').exists(): + print('ERROR: No graph found. Run /graphify first to build the graph.') + raise SystemExit(1) +" +``` +If it fails, stop and tell the user to run `/graphify ` first. + +```bash +python3 -c " +import json, sys +import networkx as nx +from networkx.readwrite import json_graph +from pathlib import Path + +data = json.loads(Path('graphify-out/graph.json').read_text()) +G = json_graph.node_link_graph(data, edges='links') + +term = 'NODE_NAME' +term_lower = term.lower() + +# Find best matching node +scored = sorted( + [(sum(1 for w in term_lower.split() if w in G.nodes[n].get('label','').lower()), n) + for n in G.nodes()], + reverse=True +) +if not scored or scored[0][0] == 0: + print(f'No node matching {term!r}') + sys.exit(0) + +nid = scored[0][1] +data_n = G.nodes[nid] +print(f'NODE: {data_n.get(\"label\", nid)}') +print(f' source: {data_n.get(\"source_file\",\"unknown\")}') +print(f' type: {data_n.get(\"file_type\",\"unknown\")}') +print(f' degree: {G.degree(nid)}') +print() +print('CONNECTIONS:') +for neighbor in G.neighbors(nid): + edge = G.edges[nid, neighbor] + nlabel = G.nodes[neighbor].get('label', neighbor) + rel = edge.get('relation', '') + conf = edge.get('confidence', '') + src_file = G.nodes[neighbor].get('source_file', '') + print(f' --{rel}--> {nlabel} [{conf}] ({src_file})') +" +``` + +Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sentence explanation of what this node is, what it connects to, and why those connections are significant. Use the source locations as citations. + +After writing the explanation, save it back: + +```bash +python3 -c " +from graphify.ingest import save_query_result +from pathlib import Path +save_query_result( + question='Explain NODE_NAME', + answer='ANSWER', + memory_dir=Path('graphify-out/memory'), + query_type='explain', + source_nodes=['NODE_NAME'], +) +print('Explanation saved to graphify-out/memory/') +" +``` + +--- + +## For /graphify add + +Fetch a URL and add it to the corpus, then update the graph. + +```bash +python3 -c " +import sys +from graphify.ingest import ingest +from pathlib import Path + +try: + out = ingest('URL', Path('./raw'), author='AUTHOR', contributor='CONTRIBUTOR') + print(f'Saved to {out}') +except ValueError as e: + print(f'error: {e}', file=sys.stderr) + sys.exit(1) +except RuntimeError as e: + print(f'error: {e}', file=sys.stderr) + sys.exit(1) +" +``` + +Replace `URL` with the actual URL, `AUTHOR` with the user's name if provided, `CONTRIBUTOR` likewise. If the command exits with an error, tell the user what went wrong - do not silently continue. After a successful save, automatically run the `--update` pipeline on `./raw` to merge the new file into the existing graph. + +Supported URL types (auto-detected): +- Twitter/X → fetched via oEmbed, saved as `.md` with tweet text and author +- arXiv → abstract + metadata saved as `.md` +- PDF → downloaded as `.pdf` +- Images (.png/.jpg/.webp) → downloaded, Claude vision extracts on next run +- Any webpage → converted to markdown via html2text + +--- + +## For --watch + +Start a background watcher that monitors a folder and auto-updates the graph when files change. + +```bash +python3 -m graphify.watch INPUT_PATH --debounce 3 +``` + +Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: + +- **Code files only (.py, .ts, .go, etc.):** re-runs AST extraction + rebuild + cluster immediately, no LLM needed. `graph.json` and `GRAPH_REPORT.md` are updated automatically. +- **Docs, papers, or images:** writes a `graphify-out/needs_update` flag and prints a notification to run `/graphify --update` (LLM semantic re-extraction required). + +Debounce (default 3s): waits until file activity stops before triggering, so a wave of parallel agent writes doesn't trigger a rebuild per file. + +Press Ctrl+C to stop. + +For agentic workflows: run `--watch` in a background terminal. Code changes from agent waves are picked up automatically between waves. If agents are also writing docs or notes, you'll need a manual `/graphify --update` after those waves. + +--- + +## For git commit hook + +Install a post-commit hook that auto-rebuilds the graph after every commit. No background process needed - triggers once per commit, works with any editor. + +```bash +graphify hook install # install +graphify hook uninstall # remove +graphify hook status # check +``` + +After every `git commit`, the hook detects which code files changed (via `git diff HEAD~1`), re-runs AST extraction on those files, and rebuilds `graph.json` and `GRAPH_REPORT.md`. Doc/image changes are ignored by the hook - run `/graphify --update` manually for those. + +If a post-commit hook already exists, graphify appends to it rather than replacing it. + +--- + +## For native CLAUDE.md integration + +Run once per project to make graphify always-on in Claude Code sessions: + +```bash +graphify claude install +``` + +This writes a `## graphify` section to the local `CLAUDE.md` that instructs Claude to check the graph before answering codebase questions and rebuild it after code changes. No manual `/graphify` needed in future sessions. + +```bash +graphify claude uninstall # remove the section +``` + +--- + +## Honesty Rules + +- Never invent an edge. If unsure, use AMBIGUOUS. +- Never skip the corpus check warning. +- Always show token cost in the report. +- Never hide cohesion scores behind symbols - show the raw number. +- Never run HTML viz on a graph with more than 5,000 nodes without warning the user. + +---HTTP_STATUS:200--- \ No newline at end of file diff --git a/tool-results/read_1793214678091_6360aa32baf6.txt b/tool-results/read_1793214678091_6360aa32baf6.txt new file mode 100644 index 0000000..7990aca --- /dev/null +++ b/tool-results/read_1793214678091_6360aa32baf6.txt @@ -0,0 +1,1206 @@ + 1→--- + 2→name: graphify + 3→description: any input (code, docs, papers, images) → knowledge graph → clustered communities → HTML + JSON + audit report + 4→trigger: /graphify + 5→--- + 6→ + 7→# /graphify + 8→ + 9→Turn any folder of files into a navigable knowledge graph with community detection, an honest audit trail, and three outputs: interactive HTML, GraphRAG-ready JSON, and a plain-language GRAPH_REPORT.md. + 10→ + 11→## Usage + 12→ + 13→``` + 14→/graphify # full pipeline on current directory → Obsidian vault + 15→/graphify # full pipeline on specific path + 16→/graphify --mode deep # thorough extraction, richer INFERRED edges + 17→/graphify --update # incremental - re-extract only new/changed files + 18→/graphify --cluster-only # rerun clustering on existing graph + 19→/graphify --no-viz # skip visualization, just report + JSON + 20→/graphify --html # (HTML is generated by default - this flag is a no-op) + 21→/graphify --svg # also export graph.svg (embeds in Notion, GitHub) + 22→/graphify --graphml # export graph.graphml (Gephi, yEd) + 23→/graphify --neo4j # generate graphify-out/cypher.txt for Neo4j + 24→/graphify --neo4j-push bolt://localhost:7687 # push directly to Neo4j + 25→/graphify --mcp # start MCP stdio server for agent access + 26→/graphify --watch # watch folder, auto-rebuild on code changes (no LLM needed) + 27→/graphify add # fetch URL, save to ./raw, update graph + 28→/graphify add --author "Name" # tag who wrote it + 29→/graphify add --contributor "Name" # tag who added it to the corpus + 30→/graphify query "" # BFS traversal - broad context + 31→/graphify query "" --dfs # DFS - trace a specific path + 32→/graphify query "" --budget 1500 # cap answer at N tokens + 33→/graphify path "AuthModule" "Database" # shortest path between two concepts + 34→/graphify explain "SwinTransformer" # plain-language explanation of a node + 35→``` + 36→ + 37→## What graphify is for + 38→ + 39→graphify is built around Andrej Karpathy's /raw folder workflow: drop anything into a folder - papers, tweets, screenshots, code, notes - and get a structured knowledge graph that shows you what you didn't know was connected. + 40→ + 41→Three things it does that Claude alone cannot: + 42→1. **Persistent graph** - relationships are stored in `graphify-out/graph.json` and survive across sessions. Ask questions weeks later without re-reading everything. + 43→2. **Honest audit trail** - every edge is tagged EXTRACTED, INFERRED, or AMBIGUOUS. You know what was found vs invented. + 44→3. **Cross-document surprise** - community detection finds connections between concepts in different files that you would never think to ask about directly. + 45→ + 46→Use it for: + 47→- A codebase you're new to (understand architecture before touching anything) + 48→- A reading list (papers + tweets + notes → one navigable graph) + 49→- A research corpus (citation graph + concept graph in one) + 50→- Your personal /raw folder (drop everything in, let it grow, query it) + 51→ + 52→## What You Must Do When Invoked + 53→ + 54→If no path was given, use `.` (current directory). Do not ask the user for a path. + 55→ + 56→Follow these steps in order. Do not skip steps. + 57→ + 58→### Step 1 - Ensure graphify is installed + 59→ + 60→```bash + 61→python3 -c "import graphify" 2>/dev/null || pip install graphifyy -q --break-system-packages 2>&1 | tail -3 + 62→``` + 63→ + 64→If the import succeeds, print nothing and move straight to Step 2. + 65→ + 66→### Step 2 - Detect files + 67→ + 68→```bash + 69→python3 -c " + 70→import json + 71→from graphify.detect import detect + 72→from pathlib import Path + 73→result = detect(Path('INPUT_PATH')) + 74→print(json.dumps(result)) + 75→" > .graphify_detect.json + 76→``` + 77→ + 78→Replace INPUT_PATH with the actual path the user provided. Do NOT cat or print the JSON - read it silently and present a clean summary instead: + 79→ + 80→``` + 81→Corpus: X files · ~Y words + 82→ code: N files (.py .ts .go ...) + 83→ docs: N files (.md .txt ...) + 84→ papers: N files (.pdf ...) + 85→ images: N files + 86→``` + 87→ + 88→Then act on it: + 89→- If `total_files` is 0: stop with "No supported files found in [path]." + 90→- If `skipped_sensitive` is non-empty: mention file count skipped, not the file names. + 91→- If `total_words` > 2,000,000 OR `total_files` > 200: show the warning and the top 5 subdirectories by file count, then ask which subfolder to run on. Wait for the user's answer before proceeding. + 92→- Otherwise: proceed directly to Step 3 - no need to ask anything. + 93→ + 94→### Step 3 - Extract entities and relationships + 95→ + 96→**Before starting:** note whether `--mode deep` was given. You must pass `DEEP_MODE=true` to every subagent in Step B2 if it was. Track this from the original invocation - do not lose it. + 97→ + 98→This step has two parts: **structural extraction** (deterministic, free) and **semantic extraction** (Claude, costs tokens). + 99→ + 100→**Run Part A (AST) and Part B (semantic) in parallel. Dispatch all semantic subagents AND start AST extraction in the same message. Both can run simultaneously since they operate on different file types. Merge results in Part C as before.** + 101→ + 102→Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is deterministic and fast; start it while subagents are processing docs/papers. + 103→ + 104→#### Part A - Structural extraction for code files + 105→ + 106→For any code files detected, run AST extraction in parallel with Part B subagents: + 107→ + 108→```bash + 109→python3 -c " + 110→import sys, json + 111→from graphify.extract import collect_files, extract + 112→from pathlib import Path + 113→import json + 114→ + 115→code_files = [] + 116→detect = json.loads(Path('.graphify_detect.json').read_text()) + 117→for f in detect.get('files', {}).get('code', []): + 118→ code_files.extend(collect_files(Path(f)) if Path(f).is_dir() else [Path(f)]) + 119→ + 120→if code_files: + 121→ result = extract(code_files) + 122→ Path('.graphify_ast.json').write_text(json.dumps(result, indent=2)) + 123→ print(f'AST: {len(result[\"nodes\"])} nodes, {len(result[\"edges\"])} edges') + 124→else: + 125→ Path('.graphify_ast.json').write_text(json.dumps({'nodes':[],'edges':[],'input_tokens':0,'output_tokens':0})) + 126→ print('No code files - skipping AST extraction') + 127→" + 128→``` + 129→ + 130→#### Part B - Semantic extraction (parallel subagents) + 131→ + 132→**Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. + 133→ + 134→**MANDATORY: You MUST use the Agent tool here. Reading files yourself one-by-one is forbidden - it is 5-10x slower. If you do not use the Agent tool you are doing this wrong.** + 135→ + 136→Before dispatching subagents, print a timing estimate: + 137→- Load `total_words` and file counts from `.graphify_detect.json` + 138→- Estimate agents needed: `ceil(uncached_non_code_files / 22)` (chunk size is 20-25) + 139→- Estimate time: ~45s per agent batch (they run in parallel, so total ≈ 45s × ceil(agents/parallel_limit)) + 140→- Print: "Semantic extraction: ~N files → X agents, estimated ~Ys" + 141→ + 142→**Step B0 - Check extraction cache first** + 143→ + 144→Before dispatching any subagents, check which files already have cached extraction results: + 145→ + 146→```bash + 147→python3 -c " + 148→import json + 149→from graphify.cache import check_semantic_cache + 150→from pathlib import Path + 151→ + 152→detect = json.loads(Path('.graphify_detect.json').read_text()) + 153→all_files = [f for files in detect['files'].values() for f in files] + 154→ + 155→cached_nodes, cached_edges, uncached = check_semantic_cache(all_files) + 156→ + 157→if cached_nodes or cached_edges: + 158→ Path('.graphify_cached.json').write_text(json.dumps({'nodes': cached_nodes, 'edges': cached_edges})) + 159→Path('.graphify_uncached.txt').write_text('\n'.join(uncached)) + 160→print(f'Cache: {len(all_files)-len(uncached)} files hit, {len(uncached)} files need extraction') + 161→" + 162→``` + 163→ + 164→Only dispatch subagents for files listed in `.graphify_uncached.txt`. If all files are cached, skip to Part C directly. + 165→ + 166→**Step B1 - Split into chunks** + 167→ + 168→Load files from `.graphify_uncached.txt`. Split into chunks of 20-25 files each. Each image gets its own chunk (vision needs separate context). + 169→ + 170→**Step B2 - Dispatch ALL subagents in a single message** + 171→ + 172→Call the Agent tool multiple times IN THE SAME RESPONSE - one call per chunk. This is the only way they run in parallel. If you make one Agent call, wait, then make another, you are doing it sequentially and defeating the purpose. + 173→ + 174→Concrete example for 3 chunks: + 175→``` + 176→[Agent tool call 1: files 1-15] + 177→[Agent tool call 2: files 16-30] + 178→[Agent tool call 3: files 31-45] + 179→``` + 180→All three in one message. Not three separate messages. + 181→ + 182→Each subagent receives this exact prompt (substitute FILE_LIST, CHUNK_NUM, TOTAL_CHUNKS, and DEEP_MODE): + 183→ + 184→``` + 185→You are a graphify extraction subagent. Read the files listed and extract a knowledge graph fragment. + 186→Output ONLY valid JSON matching the schema below - no explanation, no markdown fences, no preamble. + 187→ + 188→Files (chunk CHUNK_NUM of TOTAL_CHUNKS): + 189→FILE_LIST + 190→ + 191→Rules: + 192→- EXTRACTED: relationship explicit in source (import, call, citation, "see §3.2") + 193→- INFERRED: reasonable inference (shared data structure, implied dependency) + 194→- AMBIGUOUS: uncertain - flag for review, do not omit + 195→ + 196→Code files: focus on semantic edges AST cannot find (call relationships, shared data, arch patterns). + 197→ Do not re-extract imports - AST already has those. + 198→Doc/paper files: extract named concepts, entities, citations. + 199→Image files: use vision to understand what the image IS - do not just OCR. + 200→ UI screenshot: layout patterns, design decisions, key elements, purpose. + 201→ Chart: metric, trend/insight, data source. + 202→ Tweet/post: claim as node, author, concepts mentioned. + 203→ Diagram: components and connections. + 204→ Research figure: what it demonstrates, method, result. + 205→ Handwritten/whiteboard: ideas and arrows, mark uncertain readings AMBIGUOUS. + 206→ + 207→DEEP_MODE (if --mode deep was given): be aggressive with INFERRED edges - indirect deps, + 208→ shared assumptions, latent couplings. Mark uncertain ones AMBIGUOUS instead of omitting. + 209→ + 210→Semantic similarity: if two concepts in this chunk solve the same problem or represent the same idea without any structural link (no import, no call, no citation), add a `semantically_similar_to` edge marked INFERRED with a confidence_score reflecting how similar they are (0.6-0.95). Examples: + 211→- Two functions that both validate user input but never call each other + 212→- A class in code and a concept in a paper that describe the same algorithm + 213→- Two error types that handle the same failure mode differently + 214→Only add these when the similarity is genuinely non-obvious and cross-cutting. Do not add them for trivially similar things. + 215→ + 216→Hyperedges: if 3 or more nodes clearly participate together in a shared concept, flow, or pattern that is not captured by pairwise edges alone, add a hyperedge to a top-level `hyperedges` array. Examples: + 217→- All classes that implement a common protocol or interface + 218→- All functions in an authentication flow (even if they don't all call each other) + 219→- All concepts from a paper section that form one coherent idea + 220→Use sparingly — only when the group relationship adds information beyond the pairwise edges. Maximum 3 hyperedges per chunk. + 221→ + 222→If a file has YAML frontmatter (--- ... ---), copy source_url, captured_at, author, + 223→ contributor onto every node from that file. + 224→ + 225→confidence_score rules: + 226→- EXTRACTED edges: confidence_score must be 1.0 + 227→- INFERRED edges: score 0.4-0.9 based on how certain you are. + 228→ Strong structural inference (e.g. two classes clearly share data): 0.8-0.9. + 229→ Reasonable but not certain: 0.6-0.7. Weak inference: 0.4-0.5. + 230→- AMBIGUOUS edges: score 0.1-0.3 + 231→ + 232→Output exactly this JSON (no other text): + 233→{"nodes":[{"id":"filestem_entityname","label":"Human Readable Name","file_type":"code|document|paper|image","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0} + 234→``` + 235→ + 236→**Step B3 - Collect, cache, and merge** + 237→ + 238→Wait for all subagents. For each result: + 239→- If a subagent returned valid JSON with `nodes` and `edges`, include it and save each file's nodes/edges to the cache + 240→- If a subagent failed or returned invalid JSON, print a warning and skip that chunk - do not abort + 241→ + 242→If more than half the chunks failed, stop and tell the user. + 243→ + 244→Save new results to cache: + 245→```bash + 246→python3 -c " + 247→import json + 248→from graphify.cache import save_semantic_cache + 249→from pathlib import Path + 250→ + 251→new = json.loads(Path('.graphify_semantic_new.json').read_text()) if Path('.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[]} + 252→saved = save_semantic_cache(new.get('nodes', []), new.get('edges', [])) + 253→print(f'Cached {saved} files') + 254→" + 255→``` + 256→ + 257→Merge cached + new results into `.graphify_semantic.json`: + 258→```bash + 259→python3 -c " + 260→import json + 261→from pathlib import Path + 262→ + 263→cached = json.loads(Path('.graphify_cached.json').read_text()) if Path('.graphify_cached.json').exists() else {'nodes':[],'edges':[]} + 264→new = json.loads(Path('.graphify_semantic_new.json').read_text()) if Path('.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[]} + 265→ + 266→all_nodes = cached['nodes'] + new.get('nodes', []) + 267→all_edges = cached['edges'] + new.get('edges', []) + 268→seen = set() + 269→deduped = [] + 270→for n in all_nodes: + 271→ if n['id'] not in seen: + 272→ seen.add(n['id']) + 273→ deduped.append(n) + 274→ + 275→merged = { + 276→ 'nodes': deduped, + 277→ 'edges': all_edges, + 278→ 'input_tokens': new.get('input_tokens', 0), + 279→ 'output_tokens': new.get('output_tokens', 0), + 280→} + 281→Path('.graphify_semantic.json').write_text(json.dumps(merged, indent=2)) + 282→print(f'Extraction complete - {len(deduped)} nodes, {len(all_edges)} edges ({len(cached[\"nodes\"])} from cache, {len(new.get(\"nodes\",[]))} new)') + 283→" + 284→``` + 285→Clean up temp files: `rm -f .graphify_cached.json .graphify_uncached.txt .graphify_semantic_new.json` + 286→ + 287→#### Part C - Merge AST + semantic into final extraction + 288→ + 289→```bash + 290→python3 -c " + 291→import sys, json + 292→from pathlib import Path + 293→ + 294→ast = json.loads(Path('.graphify_ast.json').read_text()) + 295→sem = json.loads(Path('.graphify_semantic.json').read_text()) + 296→ + 297→# Merge: AST nodes first, semantic nodes deduplicated by id + 298→seen = {n['id'] for n in ast['nodes']} + 299→merged_nodes = list(ast['nodes']) + 300→for n in sem['nodes']: + 301→ if n['id'] not in seen: + 302→ merged_nodes.append(n) + 303→ seen.add(n['id']) + 304→ + 305→merged_edges = ast['edges'] + sem['edges'] + 306→merged = { + 307→ 'nodes': merged_nodes, + 308→ 'edges': merged_edges, + 309→ 'input_tokens': sem.get('input_tokens', 0), + 310→ 'output_tokens': sem.get('output_tokens', 0), + 311→} + 312→Path('.graphify_extract.json').write_text(json.dumps(merged, indent=2)) + 313→total = len(merged_nodes) + 314→edges = len(merged_edges) + 315→print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(sem[\"nodes\"])} semantic)') + 316→" + 317→``` + 318→ + 319→### Step 4 - Build graph, cluster, analyze, generate outputs + 320→ + 321→```bash + 322→mkdir -p graphify-out + 323→python3 -c " + 324→import sys, json + 325→from graphify.build import build_from_json + 326→from graphify.cluster import cluster, score_all + 327→from graphify.analyze import god_nodes, surprising_connections, suggest_questions + 328→from graphify.report import generate + 329→from graphify.export import to_json + 330→from pathlib import Path + 331→ + 332→extraction = json.loads(Path('.graphify_extract.json').read_text()) + 333→detection = json.loads(Path('.graphify_detect.json').read_text()) + 334→ + 335→G = build_from_json(extraction) + 336→communities = cluster(G) + 337→cohesion = score_all(G, communities) + 338→tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} + 339→gods = god_nodes(G) + 340→surprises = surprising_connections(G, communities) + 341→labels = {cid: 'Community ' + str(cid) for cid in communities} + 342→# Placeholder questions - regenerated with real labels in Step 5 + 343→questions = suggest_questions(G, communities, labels) + 344→ + 345→report = generate(G, communities, cohesion, labels, gods, surprises, detection, tokens, 'INPUT_PATH', suggested_questions=questions) + 346→Path('graphify-out/GRAPH_REPORT.md').write_text(report) + 347→to_json(G, communities, 'graphify-out/graph.json') + 348→ + 349→analysis = { + 350→ 'communities': {str(k): v for k, v in communities.items()}, + 351→ 'cohesion': {str(k): v for k, v in cohesion.items()}, + 352→ 'gods': gods, + 353→ 'surprises': surprises, + 354→ 'questions': questions, + 355→} + 356→Path('.graphify_analysis.json').write_text(json.dumps(analysis, indent=2)) + 357→if G.number_of_nodes() == 0: + 358→ print('ERROR: Graph is empty - extraction produced no nodes.') + 359→ print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') + 360→ raise SystemExit(1) + 361→print(f'Graph: {G.number_of_nodes()} nodes, {G.number_of_edges()} edges, {len(communities)} communities') + 362→" + 363→``` + 364→ + 365→If this step prints `ERROR: Graph is empty`, stop and tell the user what happened - do not proceed to labeling or visualization. + 366→ + 367→Replace INPUT_PATH with the actual path. + 368→ + 369→### Step 5 - Label communities + 370→ + 371→Read `.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). + 372→ + 373→Then regenerate the report and save the labels for the visualizer: + 374→ + 375→```bash + 376→python3 -c " + 377→import sys, json + 378→from graphify.build import build_from_json + 379→from graphify.cluster import score_all + 380→from graphify.analyze import god_nodes, surprising_connections, suggest_questions + 381→from graphify.report import generate + 382→from pathlib import Path + 383→ + 384→extraction = json.loads(Path('.graphify_extract.json').read_text()) + 385→detection = json.loads(Path('.graphify_detect.json').read_text()) + 386→analysis = json.loads(Path('.graphify_analysis.json').read_text()) + 387→ + 388→G = build_from_json(extraction) + 389→communities = {int(k): v for k, v in analysis['communities'].items()} + 390→cohesion = {int(k): v for k, v in analysis['cohesion'].items()} + 391→tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} + 392→ + 393→# LABELS - replace these with the names you chose above + 394→labels = LABELS_DICT + 395→ + 396→# Regenerate questions with real community labels (labels affect question phrasing) + 397→questions = suggest_questions(G, communities, labels) + 398→ + 399→report = generate(G, communities, cohesion, labels, analysis['gods'], analysis['surprises'], detection, tokens, 'INPUT_PATH', suggested_questions=questions) + 400→Path('graphify-out/GRAPH_REPORT.md').write_text(report) + 401→Path('.graphify_labels.json').write_text(json.dumps({str(k): v for k, v in labels.items()})) + 402→print('Report updated with community labels') + 403→" + 404→``` + 405→ + 406→Replace `LABELS_DICT` with the actual dict you constructed (e.g. `{0: "Attention Mechanism", 1: "Training Pipeline"}`). + 407→Replace INPUT_PATH with the actual path. + 408→ + 409→### Step 6 - Generate Obsidian vault (default) + optional HTML + 410→ + 411→**Always generate the Obsidian vault and HTML** - they are the primary visualizations. Skip both if `--no-viz` (report + JSON only). + 412→ + 413→```bash + 414→python3 -c " + 415→import sys, json + 416→from graphify.build import build_from_json + 417→from graphify.export import to_obsidian, to_canvas + 418→from pathlib import Path + 419→ + 420→extraction = json.loads(Path('.graphify_extract.json').read_text()) + 421→analysis = json.loads(Path('.graphify_analysis.json').read_text()) + 422→labels_raw = json.loads(Path('.graphify_labels.json').read_text()) if Path('.graphify_labels.json').exists() else {} + 423→ + 424→G = build_from_json(extraction) + 425→communities = {int(k): v for k, v in analysis['communities'].items()} + 426→cohesion = {int(k): v for k, v in analysis['cohesion'].items()} + 427→labels = {int(k): v for k, v in labels_raw.items()} + 428→ + 429→n = to_obsidian(G, communities, 'graphify-out/obsidian', community_labels=labels or None, cohesion=cohesion) + 430→print(f'Obsidian vault: {n} notes in graphify-out/obsidian/') + 431→ + 432→to_canvas(G, communities, 'graphify-out/obsidian/graph.canvas', community_labels=labels or None) + 433→print('Canvas: graphify-out/obsidian/graph.canvas - open in Obsidian for structured community layout') + 434→print() + 435→print('Open graphify-out/obsidian/ as a vault in Obsidian.') + 436→print(' Graph view - nodes colored by community (set automatically)') + 437→print(' graph.canvas - structured layout with communities as groups') + 438→print(' _COMMUNITY_* - overview notes with cohesion scores and dataview queries') + 439→" + 440→``` + 441→ + 442→Also generate the HTML graph (always, unless `--no-viz`): + 443→ + 444→```bash + 445→python3 -c " + 446→import sys, json + 447→from graphify.build import build_from_json + 448→from graphify.export import to_html + 449→from pathlib import Path + 450→ + 451→extraction = json.loads(Path('.graphify_extract.json').read_text()) + 452→analysis = json.loads(Path('.graphify_analysis.json').read_text()) + 453→labels_raw = json.loads(Path('.graphify_labels.json').read_text()) if Path('.graphify_labels.json').exists() else {} + 454→ + 455→G = build_from_json(extraction) + 456→communities = {int(k): v for k, v in analysis['communities'].items()} + 457→labels = {int(k): v for k, v in labels_raw.items()} + 458→ + 459→if G.number_of_nodes() > 5000: + 460→ print(f'Graph has {G.number_of_nodes()} nodes - too large for HTML viz. Use Obsidian vault instead.') + 461→else: + 462→ to_html(G, communities, 'graphify-out/graph.html', community_labels=labels or None) + 463→ print('graph.html written - open in any browser, no server needed') + 464→" + 465→``` + 466→ + 467→### Step 7 - Neo4j export (only if --neo4j or --neo4j-push flag) + 468→ + 469→**If `--neo4j`** - generate a Cypher file for manual import: + 470→ + 471→```bash + 472→python3 -c " + 473→import sys, json + 474→from graphify.build import build_from_json + 475→from graphify.export import to_cypher + 476→from pathlib import Path + 477→ + 478→G = build_from_json(json.loads(Path('.graphify_extract.json').read_text())) + 479→to_cypher(G, 'graphify-out/cypher.txt') + 480→print('cypher.txt written - import with: cypher-shell < graphify-out/cypher.txt') + 481→" + 482→``` + 483→ + 484→**If `--neo4j-push `** - push directly to a running Neo4j instance. Ask the user for credentials if not provided: + 485→ + 486→```bash + 487→python3 -c " + 488→import sys, json + 489→from graphify.build import build_from_json + 490→from graphify.cluster import cluster + 491→from graphify.export import push_to_neo4j + 492→from pathlib import Path + 493→ + 494→extraction = json.loads(Path('.graphify_extract.json').read_text()) + 495→analysis = json.loads(Path('.graphify_analysis.json').read_text()) + 496→G = build_from_json(extraction) + 497→communities = {int(k): v for k, v in analysis['communities'].items()} + 498→ + 499→result = push_to_neo4j(G, uri='NEO4J_URI', user='NEO4J_USER', password='NEO4J_PASSWORD', communities=communities) + 500→print(f'Pushed to Neo4j: {result[\"nodes\"]} nodes, {result[\"edges\"]} edges') + 501→" + 502→``` + 503→ + 504→Replace `NEO4J_URI`, `NEO4J_USER`, `NEO4J_PASSWORD` with actual values. Default URI is `bolt://localhost:7687`, default user is `neo4j`. Uses MERGE - safe to re-run without creating duplicates. + 505→ + 506→### Step 7b - SVG export (only if --svg flag) + 507→ + 508→```bash + 509→python3 -c " + 510→import sys, json + 511→from graphify.build import build_from_json + 512→from graphify.export import to_svg + 513→from pathlib import Path + 514→ + 515→extraction = json.loads(Path('.graphify_extract.json').read_text()) + 516→analysis = json.loads(Path('.graphify_analysis.json').read_text()) + 517→labels_raw = json.loads(Path('.graphify_labels.json').read_text()) if Path('.graphify_labels.json').exists() else {} + 518→ + 519→G = build_from_json(extraction) + 520→communities = {int(k): v for k, v in analysis['communities'].items()} + 521→labels = {int(k): v for k, v in labels_raw.items()} + 522→ + 523→to_svg(G, communities, 'graphify-out/graph.svg', community_labels=labels or None) + 524→print('graph.svg written - embeds in Obsidian, Notion, GitHub READMEs') + 525→" + 526→``` + 527→ + 528→### Step 7c - GraphML export (only if --graphml flag) + 529→ + 530→```bash + 531→python3 -c " + 532→import json + 533→from graphify.build import build_from_json + 534→from graphify.export import to_graphml + 535→from pathlib import Path + 536→ + 537→extraction = json.loads(Path('.graphify_extract.json').read_text()) + 538→analysis = json.loads(Path('.graphify_analysis.json').read_text()) + 539→ + 540→G = build_from_json(extraction) + 541→communities = {int(k): v for k, v in analysis['communities'].items()} + 542→ + 543→to_graphml(G, communities, 'graphify-out/graph.graphml') + 544→print('graph.graphml written - open in Gephi, yEd, or any GraphML tool') + 545→" + 546→``` + 547→ + 548→### Step 7d - MCP server (only if --mcp flag) + 549→ + 550→```bash + 551→python3 -m graphify.serve graphify-out/graph.json + 552→``` + 553→ + 554→This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. + 555→ + 556→To configure in Claude Desktop, add to `claude_desktop_config.json`: + 557→```json + 558→{ + 559→ "mcpServers": { + 560→ "graphify": { + 561→ "command": "python3", + 562→ "args": ["-m", "graphify.serve", "/absolute/path/to/graphify-out/graph.json"] + 563→ } + 564→ } + 565→} + 566→``` + 567→ + 568→### Step 8 - Token reduction benchmark (only if total_words > 5000) + 569→ + 570→If `total_words` from `.graphify_detect.json` is greater than 5,000, run: + 571→ + 572→```bash + 573→python3 -c " + 574→import json + 575→from graphify.benchmark import run_benchmark, print_benchmark + 576→from pathlib import Path + 577→ + 578→detection = json.loads(Path('.graphify_detect.json').read_text()) + 579→result = run_benchmark('graphify-out/graph.json', corpus_words=detection['total_words']) + 580→print_benchmark(result) + 581→" + 582→``` + 583→ + 584→Print the output directly in chat. If `total_words <= 5000`, skip silently - the graph value is structural clarity, not token compression, for small corpora. + 585→ + 586→--- + 587→ + 588→### Step 9 - Save manifest, update cost tracker, clean up, and report + 589→ + 590→```bash + 591→python3 -c " + 592→import json + 593→from pathlib import Path + 594→from datetime import datetime, timezone + 595→from graphify.detect import save_manifest + 596→ + 597→# Save manifest for --update + 598→detect = json.loads(Path('.graphify_detect.json').read_text()) + 599→save_manifest(detect['files']) + 600→ + 601→# Update cumulative cost tracker + 602→extract = json.loads(Path('.graphify_extract.json').read_text()) + 603→input_tok = extract.get('input_tokens', 0) + 604→output_tok = extract.get('output_tokens', 0) + 605→ + 606→cost_path = Path('graphify-out/cost.json') + 607→if cost_path.exists(): + 608→ cost = json.loads(cost_path.read_text()) + 609→else: + 610→ cost = {'runs': [], 'total_input_tokens': 0, 'total_output_tokens': 0} + 611→ + 612→cost['runs'].append({ + 613→ 'date': datetime.now(timezone.utc).isoformat(), + 614→ 'input_tokens': input_tok, + 615→ 'output_tokens': output_tok, + 616→ 'files': detect.get('total_files', 0), + 617→}) + 618→cost['total_input_tokens'] += input_tok + 619→cost['total_output_tokens'] += output_tok + 620→cost_path.write_text(json.dumps(cost, indent=2)) + 621→ + 622→print(f'This run: {input_tok:,} input tokens, {output_tok:,} output tokens') + 623→print(f'All time: {cost[\"total_input_tokens\"]:,} input, {cost[\"total_output_tokens\"]:,} output ({len(cost[\"runs\"])} runs)') + 624→" + 625→rm -f .graphify_detect.json .graphify_extract.json .graphify_ast.json .graphify_semantic.json .graphify_analysis.json .graphify_labels.json + 626→rm -f graphify-out/.needs_update 2>/dev/null || true + 627→``` + 628→ + 629→Tell the user: + 630→``` + 631→Graph complete. Outputs are in a hidden folder called graphify-out/ inside the directory you ran this on. + 632→ + 633→The folder is hidden (dot prefix) so it won't show in Finder or a normal ls. + 634→To see it: + 635→ Mac/Linux: ls -la graphify-out/ + 636→ VS Code: the Explorer panel shows hidden files by default + 637→ Finder: Cmd+Shift+. to toggle hidden files + 638→ + 639→What's inside: + 640→ graphify-out/obsidian/ - open this folder as a vault in Obsidian (File > Open Vault) + 641→ graphify-out/GRAPH_REPORT.md - full audit report, also readable here in Claude + 642→ graphify-out/graph.json - persistent graph, query it later with /graphify query "..." + 643→ + 644→Full path: PATH_TO_DIR/graphify-out/ + 645→``` + 646→ + 647→Replace PATH_TO_DIR with the actual absolute path of the directory that was processed. + 648→ + 649→Then paste these sections from GRAPH_REPORT.md directly into the chat: + 650→- God Nodes + 651→- Surprising Connections + 652→- Suggested Questions + 653→ + 654→Do NOT paste the full report - just those three sections. Keep it concise. + 655→ + 656→Then immediately offer to explore. Pick the single most interesting suggested question from the report - the one that crosses the most community boundaries or has the most surprising bridge node - and ask: + 657→ + 658→> "The most interesting question this graph can answer: **[question]**. Want me to trace it?" + 659→ + 660→If the user says yes, run `/graphify query "[question]"` on the graph and walk them through the answer using the graph structure - which nodes connect, which community boundaries get crossed, what the path reveals. Keep going as long as they want to explore. Each answer should end with a natural follow-up ("this connects to X - want to go deeper?") so the session feels like navigation, not a one-shot report. + 661→ + 662→The graph is the map. Your job after the pipeline is to be the guide. + 663→ + 664→--- + 665→ + 666→## For --update (incremental re-extraction) + 667→ + 668→Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. + 669→ + 670→```bash + 671→python3 -c " + 672→import sys, json + 673→from graphify.detect import detect_incremental, save_manifest + 674→from pathlib import Path + 675→ + 676→result = detect_incremental(Path('INPUT_PATH')) + 677→new_total = result.get('new_total', 0) + 678→print(json.dumps(result, indent=2)) + 679→Path('.graphify_incremental.json').write_text(json.dumps(result)) + 680→if new_total == 0: + 681→ print('No files changed since last run. Nothing to update.') + 682→ raise SystemExit(0) + 683→print(f'{new_total} new/changed file(s) to re-extract.') + 684→" + 685→``` + 686→ + 687→If new files exist, first check whether all changed files are code files: + 688→ + 689→```bash + 690→python3 -c " + 691→import json + 692→from pathlib import Path + 693→ + 694→result = json.loads(open('.graphify_incremental.json').read()) if Path('.graphify_incremental.json').exists() else {} + 695→code_exts = {'.py','.ts','.js','.go','.rs','.java','.cpp','.c','.rb','.swift','.kt','.cs','.scala','.php','.cc','.cxx','.hpp','.h','.kts'} + 696→new_files = result.get('new_files', {}) + 697→all_changed = [f for files in new_files.values() for f in files] + 698→code_only = all(Path(f).suffix.lower() in code_exts for f in all_changed) + 699→print('code_only:', code_only) + 700→" + 701→``` + 702→ + 703→If `code_only` is True: print `[graphify update] Code-only changes detected - skipping semantic extraction (no LLM needed)`, run only Step 3A (AST) on the changed files, skip Step 3B entirely (no subagents), then go straight to merge and Steps 4–8. + 704→ + 705→If `code_only` is False (any changed file is a doc/paper/image): run the full Steps 3A–3C pipeline as normal. + 706→ + 707→Then: + 708→ + 709→```bash + 710→python3 -c " + 711→import sys, json + 712→from graphify.build import build_from_json + 713→from graphify.export import to_json + 714→from networkx.readwrite import json_graph + 715→import networkx as nx + 716→from pathlib import Path + 717→ + 718→# Load existing graph + 719→existing_data = json.loads(Path('graphify-out/graph.json').read_text()) + 720→G_existing = json_graph.node_link_graph(existing_data, edges='links') + 721→ + 722→# Load new extraction + 723→new_extraction = json.loads(Path('.graphify_extract.json').read_text()) + 724→G_new = build_from_json(new_extraction) + 725→ + 726→# Merge: new nodes/edges into existing graph + 727→G_existing.update(G_new) + 728→print(f'Merged: {G_existing.number_of_nodes()} nodes, {G_existing.number_of_edges()} edges') + 729→" + 730→``` + 731→ + 732→Then run Steps 4–8 on the merged graph as normal. + 733→ + 734→After Step 4, show the graph diff: + 735→ + 736→```bash + 737→python3 -c " + 738→import json + 739→from graphify.analyze import graph_diff + 740→from graphify.build import build_from_json + 741→from networkx.readwrite import json_graph + 742→import networkx as nx + 743→from pathlib import Path + 744→ + 745→# Load old graph (before update) from backup written before merge + 746→old_data = json.loads(Path('.graphify_old.json').read_text()) if Path('.graphify_old.json').exists() else None + 747→new_extract = json.loads(Path('.graphify_extract.json').read_text()) + 748→G_new = build_from_json(new_extract) + 749→ + 750→if old_data: + 751→ G_old = json_graph.node_link_graph(old_data, edges='links') + 752→ diff = graph_diff(G_old, G_new) + 753→ print(diff['summary']) + 754→ if diff['new_nodes']: + 755→ print('New nodes:', ', '.join(n['label'] for n in diff['new_nodes'][:5])) + 756→ if diff['new_edges']: + 757→ print('New edges:', len(diff['new_edges'])) + 758→" + 759→``` + 760→ + 761→Before the merge step, save the old graph: `cp graphify-out/graph.json .graphify_old.json` + 762→Clean up after: `rm -f .graphify_old.json` + 763→ + 764→--- + 765→ + 766→## For --cluster-only + 767→ + 768→Skip Steps 1–3. Load the existing graph from `graphify-out/graph.json` and re-run clustering: + 769→ + 770→```bash + 771→python3 -c " + 772→import sys, json + 773→from graphify.cluster import cluster, score_all + 774→from graphify.analyze import god_nodes, surprising_connections + 775→from graphify.report import generate + 776→from graphify.export import to_json + 777→from networkx.readwrite import json_graph + 778→import networkx as nx + 779→from pathlib import Path + 780→ + 781→data = json.loads(Path('graphify-out/graph.json').read_text()) + 782→G = json_graph.node_link_graph(data, edges='links') + 783→ + 784→detection = {'total_files': 0, 'total_words': 99999, 'needs_graph': True, 'warning': None, + 785→ 'files': {'code': [], 'document': [], 'paper': []}} + 786→tokens = {'input': 0, 'output': 0} + 787→ + 788→communities = cluster(G) + 789→cohesion = score_all(G, communities) + 790→gods = god_nodes(G) + 791→surprises = surprising_connections(G, communities) + 792→labels = {cid: 'Community ' + str(cid) for cid in communities} + 793→ + 794→report = generate(G, communities, cohesion, labels, gods, surprises, detection, tokens, '.') + 795→Path('graphify-out/GRAPH_REPORT.md').write_text(report) + 796→to_json(G, communities, 'graphify-out/graph.json') + 797→ + 798→analysis = { + 799→ 'communities': {str(k): v for k, v in communities.items()}, + 800→ 'cohesion': {str(k): v for k, v in cohesion.items()}, + 801→ 'gods': gods, + 802→ 'surprises': surprises, + 803→} + 804→Path('.graphify_analysis.json').write_text(json.dumps(analysis, indent=2)) + 805→print(f'Re-clustered: {len(communities)} communities') + 806→" + 807→``` + 808→ + 809→Then run Steps 5–9 as normal (label communities, generate viz, benchmark, clean up, report). + 810→ + 811→--- + 812→ + 813→## For /graphify query + 814→ + 815→Two traversal modes - choose based on the question: + 816→ + 817→| Mode | Flag | Best for | + 818→|------|------|----------| + 819→| BFS (default) | _(none)_ | "What is X connected to?" - broad context, nearest neighbors first | + 820→| DFS | `--dfs` | "How does X reach Y?" - trace a specific chain or dependency path | + 821→ + 822→First check the graph exists: + 823→```bash + 824→python3 -c " + 825→from pathlib import Path + 826→if not Path('graphify-out/graph.json').exists(): + 827→ print('ERROR: No graph found. Run /graphify first to build the graph.') + 828→ raise SystemExit(1) + 829→" + 830→``` + 831→If it fails, stop and tell the user to run `/graphify ` first. + 832→ + 833→Load `graphify-out/graph.json`, then: + 834→ + 835→1. Find the 1-3 nodes whose label best matches key terms in the question. + 836→2. Run the appropriate traversal from each starting node. + 837→3. Read the subgraph - node labels, edge relations, confidence tags, source locations. + 838→4. Answer using **only** what the graph contains. Quote `source_location` when citing a specific fact. + 839→5. If the graph lacks enough information, say so - do not hallucinate edges. + 840→ + 841→```bash + 842→python3 -c " + 843→import sys, json + 844→from networkx.readwrite import json_graph + 845→import networkx as nx + 846→from pathlib import Path + 847→ + 848→data = json.loads(Path('graphify-out/graph.json').read_text()) + 849→G = json_graph.node_link_graph(data, edges='links') + 850→ + 851→question = 'QUESTION' + 852→mode = 'MODE' # 'bfs' or 'dfs' + 853→terms = [t.lower() for t in question.split() if len(t) > 3] + 854→ + 855→# Find best-matching start nodes + 856→scored = [] + 857→for nid, ndata in G.nodes(data=True): + 858→ label = ndata.get('label', '').lower() + 859→ score = sum(1 for t in terms if t in label) + 860→ if score > 0: + 861→ scored.append((score, nid)) + 862→scored.sort(reverse=True) + 863→start_nodes = [nid for _, nid in scored[:3]] + 864→ + 865→if not start_nodes: + 866→ print('No matching nodes found for query terms:', terms) + 867→ sys.exit(0) + 868→ + 869→subgraph_nodes = set() + 870→subgraph_edges = [] + 871→ + 872→if mode == 'dfs': + 873→ # DFS: follow one path as deep as possible before backtracking. + 874→ # Depth-limited to 6 to avoid traversing the whole graph. + 875→ visited = set() + 876→ stack = [(n, 0) for n in reversed(start_nodes)] + 877→ while stack: + 878→ node, depth = stack.pop() + 879→ if node in visited or depth > 6: + 880→ continue + 881→ visited.add(node) + 882→ subgraph_nodes.add(node) + 883→ for neighbor in G.neighbors(node): + 884→ if neighbor not in visited: + 885→ stack.append((neighbor, depth + 1)) + 886→ subgraph_edges.append((node, neighbor)) + 887→else: + 888→ # BFS: explore all neighbors layer by layer up to depth 3. + 889→ frontier = set(start_nodes) + 890→ subgraph_nodes = set(start_nodes) + 891→ for _ in range(3): + 892→ next_frontier = set() + 893→ for n in frontier: + 894→ for neighbor in G.neighbors(n): + 895→ if neighbor not in subgraph_nodes: + 896→ next_frontier.add(neighbor) + 897→ subgraph_edges.append((n, neighbor)) + 898→ subgraph_nodes.update(next_frontier) + 899→ frontier = next_frontier + 900→ + 901→# Token-budget aware output: rank by relevance, cut at budget (~4 chars/token) + 902→token_budget = BUDGET # default 2000 + 903→char_budget = token_budget * 4 + 904→ + 905→# Score each node by term overlap for ranked output + 906→def relevance(nid): + 907→ label = G.nodes[nid].get('label', '').lower() + 908→ return sum(1 for t in terms if t in label) + 909→ + 910→ranked_nodes = sorted(subgraph_nodes, key=relevance, reverse=True) + 911→ + 912→lines = [f'Traversal: {mode.upper()} | Start: {[G.nodes[n].get(\"label\",n) for n in start_nodes]} | {len(subgraph_nodes)} nodes'] + 913→for nid in ranked_nodes: + 914→ d = G.nodes[nid] + 915→ lines.append(f' NODE {d.get(\"label\", nid)} [src={d.get(\"source_file\",\"\")} loc={d.get(\"source_location\",\"\")}]') + 916→for u, v in subgraph_edges: + 917→ if u in subgraph_nodes and v in subgraph_nodes: + 918→ d = G.edges[u, v] + 919→ lines.append(f' EDGE {G.nodes[u].get(\"label\",u)} --{d.get(\"relation\",\"\")} [{d.get(\"confidence\",\"\")}]--> {G.nodes[v].get(\"label\",v)}') + 920→ + 921→output = '\n'.join(lines) + 922→if len(output) > char_budget: + 923→ output = output[:char_budget] + f'\n... (truncated at ~{token_budget} token budget - use --budget N for more)' + 924→print(output) + 925→" + 926→``` + 927→ + 928→Replace `QUESTION` with the user's actual question, `MODE` with `bfs` or `dfs`, and `BUDGET` with the token budget (default `2000`, or whatever `--budget N` specifies). Then answer based on the subgraph output above. + 929→ + 930→After writing the answer, save it back into the graph so it improves future queries: + 931→ + 932→```bash + 933→python3 -c " + 934→from graphify.ingest import save_query_result + 935→from pathlib import Path + 936→save_query_result( + 937→ question='QUESTION', + 938→ answer='ANSWER', + 939→ memory_dir=Path('graphify-out/memory'), + 940→ query_type='query', + 941→ source_nodes=SOURCE_NODES, # list of node labels cited, or [] + 942→) + 943→print('Query result saved to graphify-out/memory/') + 944→" + 945→``` + 946→ + 947→Replace `QUESTION` with the question, `ANSWER` with your full answer text, `SOURCE_NODES` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. + 948→ + 949→--- + 950→ + 951→## For /graphify path + 952→ + 953→Find the shortest path between two named concepts in the graph. + 954→ + 955→First check the graph exists: + 956→```bash + 957→python3 -c " + 958→from pathlib import Path + 959→if not Path('graphify-out/graph.json').exists(): + 960→ print('ERROR: No graph found. Run /graphify first to build the graph.') + 961→ raise SystemExit(1) + 962→" + 963→``` + 964→If it fails, stop and tell the user to run `/graphify ` first. + 965→ + 966→```bash + 967→python3 -c " + 968→import json, sys + 969→import networkx as nx + 970→from networkx.readwrite import json_graph + 971→from pathlib import Path + 972→ + 973→data = json.loads(Path('graphify-out/graph.json').read_text()) + 974→G = json_graph.node_link_graph(data, edges='links') + 975→ + 976→a_term = 'NODE_A' + 977→b_term = 'NODE_B' + 978→ + 979→def find_node(term): + 980→ term = term.lower() + 981→ scored = sorted( + 982→ [(sum(1 for w in term.split() if w in G.nodes[n].get('label','').lower()), n) + 983→ for n in G.nodes()], + 984→ reverse=True + 985→ ) + 986→ return scored[0][1] if scored and scored[0][0] > 0 else None + 987→ + 988→src = find_node(a_term) + 989→tgt = find_node(b_term) + 990→ + 991→if not src or not tgt: + 992→ print(f'Could not find nodes matching: {a_term!r} or {b_term!r}') + 993→ sys.exit(0) + 994→ + 995→try: + 996→ path = nx.shortest_path(G, src, tgt) + 997→ print(f'Shortest path ({len(path)-1} hops):') + 998→ for i, nid in enumerate(path): + 999→ label = G.nodes[nid].get('label', nid) + 1000→ if i < len(path) - 1: + 1001→ edge = G.edges[nid, path[i+1]] + 1002→ rel = edge.get('relation', '') + 1003→ conf = edge.get('confidence', '') + 1004→ print(f' {label} --{rel}--> [{conf}]') + 1005→ else: + 1006→ print(f' {label}') + 1007→except nx.NetworkXNoPath: + 1008→ print(f'No path found between {a_term!r} and {b_term!r}') + 1009→except nx.NodeNotFound as e: + 1010→ print(f'Node not found: {e}') + 1011→" + 1012→``` + 1013→ + 1014→Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then explain the path in plain language - what each hop means, why it's significant. + 1015→ + 1016→After writing the explanation, save it back: + 1017→ + 1018→```bash + 1019→python3 -c " + 1020→from graphify.ingest import save_query_result + 1021→from pathlib import Path + 1022→save_query_result( + 1023→ question='Path from NODE_A to NODE_B', + 1024→ answer='ANSWER', + 1025→ memory_dir=Path('graphify-out/memory'), + 1026→ query_type='path_query', + 1027→ source_nodes=PATH_NODES, # list of node labels on the path + 1028→) + 1029→print('Path result saved to graphify-out/memory/') + 1030→" + 1031→``` + 1032→ + 1033→--- + 1034→ + 1035→## For /graphify explain + 1036→ + 1037→Give a plain-language explanation of a single node - everything connected to it. + 1038→ + 1039→First check the graph exists: + 1040→```bash + 1041→python3 -c " + 1042→from pathlib import Path + 1043→if not Path('graphify-out/graph.json').exists(): + 1044→ print('ERROR: No graph found. Run /graphify first to build the graph.') + 1045→ raise SystemExit(1) + 1046→" + 1047→``` + 1048→If it fails, stop and tell the user to run `/graphify ` first. + 1049→ + 1050→```bash + 1051→python3 -c " + 1052→import json, sys + 1053→import networkx as nx + 1054→from networkx.readwrite import json_graph + 1055→from pathlib import Path + 1056→ + 1057→data = json.loads(Path('graphify-out/graph.json').read_text()) + 1058→G = json_graph.node_link_graph(data, edges='links') + 1059→ + 1060→term = 'NODE_NAME' + 1061→term_lower = term.lower() + 1062→ + 1063→# Find best matching node + 1064→scored = sorted( + 1065→ [(sum(1 for w in term_lower.split() if w in G.nodes[n].get('label','').lower()), n) + 1066→ for n in G.nodes()], + 1067→ reverse=True + 1068→) + 1069→if not scored or scored[0][0] == 0: + 1070→ print(f'No node matching {term!r}') + 1071→ sys.exit(0) + 1072→ + 1073→nid = scored[0][1] + 1074→data_n = G.nodes[nid] + 1075→print(f'NODE: {data_n.get(\"label\", nid)}') + 1076→print(f' source: {data_n.get(\"source_file\",\"unknown\")}') + 1077→print(f' type: {data_n.get(\"file_type\",\"unknown\")}') + 1078→print(f' degree: {G.degree(nid)}') + 1079→print() + 1080→print('CONNECTIONS:') + 1081→for neighbor in G.neighbors(nid): + 1082→ edge = G.edges[nid, neighbor] + 1083→ nlabel = G.nodes[neighbor].get('label', neighbor) + 1084→ rel = edge.get('relation', '') + 1085→ conf = edge.get('confidence', '') + 1086→ src_file = G.nodes[neighbor].get('source_file', '') + 1087→ print(f' --{rel}--> {nlabel} [{conf}] ({src_file})') + 1088→" + 1089→``` + 1090→ + 1091→Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sentence explanation of what this node is, what it connects to, and why those connections are significant. Use the source locations as citations. + 1092→ + 1093→After writing the explanation, save it back: + 1094→ + 1095→```bash + 1096→python3 -c " + 1097→from graphify.ingest import save_query_result + 1098→from pathlib import Path + 1099→save_query_result( + 1100→ question='Explain NODE_NAME', + 1101→ answer='ANSWER', + 1102→ memory_dir=Path('graphify-out/memory'), + 1103→ query_type='explain', + 1104→ source_nodes=['NODE_NAME'], + 1105→) + 1106→print('Explanation saved to graphify-out/memory/') + 1107→" + 1108→``` + 1109→ + 1110→--- + 1111→ + 1112→## For /graphify add + 1113→ + 1114→Fetch a URL and add it to the corpus, then update the graph. + 1115→ + 1116→```bash + 1117→python3 -c " + 1118→import sys + 1119→from graphify.ingest import ingest + 1120→from pathlib import Path + 1121→ + 1122→try: + 1123→ out = ingest('URL', Path('./raw'), author='AUTHOR', contributor='CONTRIBUTOR') + 1124→ print(f'Saved to {out}') + 1125→except ValueError as e: + 1126→ print(f'error: {e}', file=sys.stderr) + 1127→ sys.exit(1) + 1128→except RuntimeError as e: + 1129→ print(f'error: {e}', file=sys.stderr) + 1130→ sys.exit(1) + 1131→" + 1132→``` + 1133→ + 1134→Replace `URL` with the actual URL, `AUTHOR` with the user's name if provided, `CONTRIBUTOR` likewise. If the command exits with an error, tell the user what went wrong - do not silently continue. After a successful save, automatically run the `--update` pipeline on `./raw` to merge the new file into the existing graph. + 1135→ + 1136→Supported URL types (auto-detected): + 1137→- Twitter/X → fetched via oEmbed, saved as `.md` with tweet text and author + 1138→- arXiv → abstract + metadata saved as `.md` + 1139→- PDF → downloaded as `.pdf` + 1140→- Images (.png/.jpg/.webp) → downloaded, Claude vision extracts on next run + 1141→- Any webpage → converted to markdown via html2text + 1142→ + 1143→--- + 1144→ + 1145→## For --watch + 1146→ + 1147→Start a background watcher that monitors a folder and auto-updates the graph when files change. + 1148→ + 1149→```bash + 1150→python3 -m graphify.watch INPUT_PATH --debounce 3 + 1151→``` + 1152→ + 1153→Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: + 1154→ + 1155→- **Code files only (.py, .ts, .go, etc.):** re-runs AST extraction + rebuild + cluster immediately, no LLM needed. `graph.json` and `GRAPH_REPORT.md` are updated automatically. + 1156→- **Docs, papers, or images:** writes a `graphify-out/needs_update` flag and prints a notification to run `/graphify --update` (LLM semantic re-extraction required). + 1157→ + 1158→Debounce (default 3s): waits until file activity stops before triggering, so a wave of parallel agent writes doesn't trigger a rebuild per file. + 1159→ + 1160→Press Ctrl+C to stop. + 1161→ + 1162→For agentic workflows: run `--watch` in a background terminal. Code changes from agent waves are picked up automatically between waves. If agents are also writing docs or notes, you'll need a manual `/graphify --update` after those waves. + 1163→ + 1164→--- + 1165→ + 1166→## For git commit hook + 1167→ + 1168→Install a post-commit hook that auto-rebuilds the graph after every commit. No background process needed - triggers once per commit, works with any editor. + 1169→ + 1170→```bash + 1171→graphify hook install # install + 1172→graphify hook uninstall # remove + 1173→graphify hook status # check + 1174→``` + 1175→ + 1176→After every `git commit`, the hook detects which code files changed (via `git diff HEAD~1`), re-runs AST extraction on those files, and rebuilds `graph.json` and `GRAPH_REPORT.md`. Doc/image changes are ignored by the hook - run `/graphify --update` manually for those. + 1177→ + 1178→If a post-commit hook already exists, graphify appends to it rather than replacing it. + 1179→ + 1180→--- + 1181→ + 1182→## For native CLAUDE.md integration + 1183→ + 1184→Run once per project to make graphify always-on in Claude Code sessions: + 1185→ + 1186→```bash + 1187→graphify claude install + 1188→``` + 1189→ + 1190→This writes a `## graphify` section to the local `CLAUDE.md` that instructs Claude to check the graph before answering codebase questions and rebuild it after code changes. No manual `/graphify` needed in future sessions. + 1191→ + 1192→```bash + 1193→graphify claude uninstall # remove the section + 1194→``` + 1195→ + 1196→--- + 1197→ + 1198→## Honesty Rules + 1199→ + 1200→- Never invent an edge. If unsure, use AMBIGUOUS. + 1201→- Never skip the corpus check warning. + 1202→- Always show token cost in the report. + 1203→- Never hide cohesion scores behind symbols - show the raw number. + 1204→- Never run HTML viz on a graph with more than 5,000 nodes without warning the user. + 1205→ + 1206→---HTTP_STATUS:200--- \ No newline at end of file diff --git a/tool-results/read_1793214682297_0713917be83f.txt b/tool-results/read_1793214682297_0713917be83f.txt new file mode 100644 index 0000000..16b43fd --- /dev/null +++ b/tool-results/read_1793214682297_0713917be83f.txt @@ -0,0 +1,517 @@ + 1→

+ 2→ Graphify + 3→

+ 4→ + 5→

+ 6→ Graphify-Labs%2Fgraphify | Trendshift + 7→

+ 8→ + 9→ + 16→ + 17→

+ 18→ PyPI + 19→ CI + 20→ License: Apache-2.0 + 21→ Downloads + 22→ Docs + 23→ Graphify Cloud + 24→ Discord + 25→ YouTube + 26→ LinkedIn + 27→ YC S26 + 28→

+ 29→ + 30→

+ 31→ Cut agent token spend by 80%. Graphify Cloud is always-on memory that governs your agents, keeps code quality high, and proves correctness with formal verification. Start a free 14-day trial → + 32→

+ 33→ + 34→Type `/graphify` in your AI coding assistant and it maps your entire project (code, docs, PDFs, images, videos) into a **knowledge graph** you can **query instead of grepping** through files. + 35→ + 36→- **Code maps for free, fully local.** Code is parsed with tree-sitter AST: deterministic, no LLM, nothing leaves your machine. (Docs, PDFs, images and video use your assistant's model, or a configured API key, for a semantic pass.) + 37→- **Every edge is explained.** Each connection is tagged `EXTRACTED` (explicit in the source) or `INFERRED` (resolved by graphify), so you can tell what was read directly from what was inferred. + 38→- **Not a vector index.** No embeddings, no vector store: a real graph you traverse. Ask a question, trace the path between two things, or explain one concept. + 39→ + 40→> [!NOTE] + 41→> **Want this always-on?** [Graphify Cloud](https://app.graphify.com/login) keeps the graph live across your entire SDLC: monorepo and cross-repo support, formal verification, and code review with a bird's-eye view of your SDLC, plus connectors for Sentry, Jira, and more so incidents and tickets sit in the same graph as your code. **[Start a free 14-day trial at app.graphify.com](https://app.graphify.com/login)**. + 42→ + 43→

+ 44→ graphify's interactive graph.html showing the FastAPI codebase as a force-directed knowledge graph with a legend of detected communities + 45→

+ 46→

+ 47→ The FastAPI codebase mapped by graphify. Every node is a concept, colors are detected communities, and the whole thing is clickable in graph.html. + 48→

+ 49→ + 50→**Get started** (30 seconds). Needs Python 3.10+ and uv (or pipx): + 51→ + 52→```bash + 53→uv tool install graphifyy # install the CLI (or: pipx install graphifyy) + 54→graphify install # detects your assistant and registers the skill + 55→``` + 56→ + 57→Then, in your AI assistant: + 58→ + 59→``` + 60→/graphify . + 61→``` + 62→ + 63→That's it. You get **three files**: + 64→ + 65→``` + 66→graphify-out/ + 67→├── graph.html open in any browser: click nodes, filter, search + 68→├── GRAPH_REPORT.md the highlights: key concepts, surprising connections, suggested questions + 69→└── graph.json the full graph: query it anytime without re-reading your files + 70→``` + 71→ + 72→The persisted graph includes `graph.schema_version` so integrations can detect + 73→incompatible format changes, plus `graph.graphify_version` identifying the + 74→Graphify release that produced it. + 75→ + 76→**One install, any assistant, any OS.** Claude Code, Cursor, Codex, Gemini CLI, GitHub Copilot, and 20+ more, on macOS, Linux, and Windows: [pick your platform](#install). + 77→ + 78→--- + 79→ + 80→## See it in action + 81→ + 82→

+ 83→ graphify path query: a terminal asks for the shortest path between FastAPI and ModelField, and the answer lights up hop by hop across the knowledge graph + 84→

+ 85→ + 86→Once the graph is built you query it instead of reading files. Real output, graphify run on the FastAPI codebase shown above: + 87→ + 88→```text + 89→$ graphify explain "APIRouter" + 90→Node: APIRouter + 91→ Source: routing.py L2210 + 92→ Community: 2 + 93→ Degree: 47 + 94→ + 95→Connections (47): + 96→ --> RequestValidationError [uses] [INFERRED] + 97→ --> Dependant [uses] [INFERRED] + 98→ --> .get() [method] [EXTRACTED] + 99→ <-- __init__.py [imports] [EXTRACTED] + 100→ ... + 101→ + 102→$ graphify path "FastAPI" "ModelField" + 103→Shortest path (3 hops): + 104→ FastAPI --uses--> DefaultPlaceholder <--references-- get_request_handler() --references--> ModelField + 105→``` + 106→ + 107→Every edge carries a **confidence tag** (`EXTRACTED` = explicit in the source, `INFERRED` = derived by resolution), so you can tell what was read directly from what was inferred. `graphify query ""` returns a scoped subgraph for a plain-language question, and `graphify path A B` traces how any two things connect. + 108→ + 109→--- + 110→ + 111→## What it does + 112→ + 113→What you get out of the box: + 114→ + 115→| Capability | What you get | + 116→|---|---| + 117→| **God nodes** | The most-connected concepts, so you see what everything flows through | + 118→| **Communities** | The graph split into subsystems (Leiden), with LLM-free labels | + 119→| **Cross-file links** | `calls` / `imports` / `inherits` / `mixes_in` resolved across ~40 languages via tree-sitter AST | + 120→| **Query, path, explain** | Ask a question, trace the path between two things, or explain one concept, all against `graph.json` | + 121→| **Rationale + doc refs** | `# NOTE:` / `# WHY:` comments and ADR/RFC citations become first-class nodes linked to the code | + 122→| **Beyond code** | Docs, PDFs, images, and video/audio all map into the same graph | + 123→| **Local-first** | Code is parsed locally with tree-sitter (no LLM, nothing leaves your machine); only the semantic pass over docs/media calls a backend, and only if you configure one | + 124→ + 125→> [!TIP] + 126→> Running this across many repos or a monorepo? [Graphify Cloud](https://app.graphify.com/login) does it continuously and cuts agent token spend by 80%, with cross-repo links, SDLC-wide code review, formal verification, and Sentry/Jira connectors. [Start a free 14-day trial →](https://app.graphify.com/login) + 127→ + 128→--- + 129→ + 130→## Benchmarks: 0.497 recall@10, 0 LLM credits + 131→ + 132→| Benchmark | Metric | graphify | Field | + 133→|---|---|---|---| + 134→| LOCOMO (n=300) | recall@10 | **0.497** | mem0 0.048, supermemory 0.149 | + 135→| LOCOMO (n=300) | QA accuracy | 45.3% | supermemory 49.7%, mem0 27.3% | + 136→| LongMemEval-S (n=50) | QA accuracy | **76%** | tied with dense RAG | + 137→| ERPNext cross-tool (n=6) | key-fact coverage | **82.0%** | grep/read baseline 70.8% | + 138→| Graph build | LLM credits | **0** | per-token for most systems | + 139→ + 140→Every system ran on the same harness with the same model and budgets, scored by a judge blind-validated against a second judge (90.6% agreement, Cohen's kappa 0.81). Full per-system tables, the code-intelligence result, and reproduction commands: **[BENCHMARKS.md](./BENCHMARKS.md)**. + 141→ + 142→--- + 143→ + 144→## Documentation + 145→ + 146→Full guides and reference live at **[docs.graphify.com](https://docs.graphify.com)**: + 147→ + 148→- **[Quickstart](https://docs.graphify.com/quickstart)**: build your first graph + 149→- **[CLI reference](https://docs.graphify.com/reference/overview)**: every command and flag + 150→- **[Ask better graph questions](https://docs.graphify.com/guides/querying)**: querying patterns + 151→- **[Configuration](https://docs.graphify.com/reference/configuration)**: environment variables and tuning + 152→- **[Supported inputs](https://docs.graphify.com/concepts/supported-inputs)**: languages and file types + 153→- **[Team workflows](https://docs.graphify.com/guides/team-workflows)** and **[PR review](https://docs.graphify.com/guides/pull-requests)** + 154→- **[Troubleshooting](https://docs.graphify.com/troubleshooting)** and **[How it works](https://docs.graphify.com/concepts/architecture)** + 155→ + 156→--- + 157→ + 158→## Prerequisites + 159→ + 160→| Requirement | Minimum | Check | Install | + 161→|---|---|---|---| + 162→| Python | 3.10+ | `python --version` | [python.org](https://www.python.org/downloads/) | + 163→| uv *(recommended)* | any | `uv --version` | `curl -LsSf https://astral.sh/uv/install.sh \| sh` | + 164→| pipx *(alternative)* | any | `pipx --version` | `pip install pipx` | + 165→ + 166→**macOS quick install (Homebrew):** + 167→```bash + 168→brew install python@3.12 uv + 169→``` + 170→ + 171→**Windows quick install:** + 172→```powershell + 173→winget install astral-sh.uv + 174→``` + 175→ + 176→**Ubuntu/Debian:** + 177→```bash + 178→sudo apt install python3.12 python3-pip pipx + 179→# or install uv: + 180→curl -LsSf https://astral.sh/uv/install.sh | sh + 181→``` + 182→ + 183→--- + 184→ + 185→## Install + 186→ + 187→> [!IMPORTANT] + 188→> **Official package:** The PyPI package is `graphifyy` (double-y). Other `graphify*` packages on PyPI are not affiliated. The CLI command is still `graphify`. + 189→ + 190→The official source repository is [Graphify-Labs/graphify](https://github.com/Graphify-Labs/graphify). + 191→ + 192→The two-command quickstart is at the top of this README. This section covers the install methods, project-scoped installs, and every supported platform. + 193→ + 194→**Install methods** (pick one): + 195→ + 196→```bash + 197→uv tool install graphifyy # recommended, isolated env (if 'graphify' isn't found after, run: uv tool update-shell) + 198→pipx install graphifyy # alternative + 199→pip install graphifyy # may need PATH setup, see Troubleshooting below + 200→``` + 201→ + 202→Then run `graphify install` once. It detects which assistant you run and registers the skill, so there is no per-assistant setup. Open your AI assistant and type `/graphify .` + 203→ + 204→To install the assistant skill into the current repository instead of your user + 205→profile, add `--project`: + 206→ + 207→```bash + 208→graphify install --project + 209→graphify install --project --platform codex + 210→``` + 211→ + 212→Project-scoped installs write under the current directory, for example + 213→`.claude/skills/graphify/SKILL.md` or `.agents/skills/graphify/SKILL.md` (plus a + 214→`references/` sidecar the skill loads on demand), and + 215→print a `git add` hint for files that can be committed. + 216→Per-platform commands that support project-scoped installs accept the same flag, + 217→for example `graphify claude install --project` or `graphify codex install --project`. + 218→ + 219→> [!TIP] + 220→> Hitting `command not found`, a `pip`-on-Mac/Windows issue, PowerShell quoting, `uvx` usage, git-hook PATH quirks, or strict mode? See **[Installation](https://docs.graphify.com/installation)** and **[Troubleshooting](https://docs.graphify.com/troubleshooting)**. + 221→ + 222→
+ 223→Pick your platform (20+ assistants, click to expand) + 224→ + 225→| Platform | Install command | + 226→|----------|----------------| + 227→| Claude Code (Linux/Mac) | `graphify install` | + 228→| Claude Code (Windows) | `graphify install` (auto-detected) or `graphify install --platform windows` | + 229→| CodeBuddy | `graphify install --platform codebuddy` | + 230→| Codex | `graphify install --platform codex` | + 231→| OpenCode | `graphify install --platform opencode` | + 232→| Kilo Code | `graphify install --platform kilo` | + 233→| GitHub Copilot CLI | `graphify install --platform copilot` | + 234→| VS Code Copilot Chat | `graphify vscode install` | + 235→| Aider | `graphify install --platform aider` | + 236→| OpenClaw | `graphify install --platform claw` | + 237→| Factory Droid | `graphify install --platform droid` | + 238→| Trae | `graphify install --platform trae` | + 239→| Trae CN | `graphify install --platform trae-cn` | + 240→| Gemini CLI | `graphify install --platform gemini` | + 241→| Hermes | `graphify install --platform hermes` | + 242→| Kimi Code | `graphify install --platform kimi` | + 243→| Amp | `graphify amp install` | + 244→| Agent Skills (cross-framework) | `graphify install --platform agents` (alias `--platform skills`) | + 245→| Kiro IDE/CLI | `graphify kiro install` | + 246→| Pi coding agent | `graphify install --platform pi`, plus `graphify pi guard install` for the native guard | + 247→| Cursor | `graphify cursor install` | + 248→| Devin CLI | `graphify devin install` | + 249→| Google Antigravity | `graphify antigravity install` | + 250→| Google Antigravity (Windows) | `graphify antigravity install` (auto-detected on Windows) | + 251→ + 252→Codex users also need `multi_agent = true` under `[features]` in `~/.codex/config.toml` for parallel extraction. CodeBuddy uses the same Agent tool and PreToolUse hook mechanism as Claude Code. Factory Droid uses the `Task` tool for parallel subagent dispatch. OpenClaw and Aider use sequential extraction (parallel agent support is still early on those platforms). Trae uses the Agent tool for parallel subagent dispatch and does **not** support `PreToolUse` hooks, so AGENTS.md is the always-on mechanism. + 253→ + 254→`--platform agents` (alias `--platform skills`) targets the generic cross-framework [Agent-Skills](https://github.com/anthropics/skills) locations: the spec's user-global `~/.agents/skills/` (read by `npx skills` and spec-compliant frameworks) for a global install, and `./.agents/skills/` for a project (`--project`) install. The bare `graphify install` stays single-platform (Claude Code) by design: use the named `agents` platform when you want the skill discoverable by any framework that reads `.agents/skills`. + 255→ + 256→Pi has no `PreToolUse` hook, so `graphify pi guard install` registers a native Pi extension (via Pi's own `pi install `) that runs the installed `graphify hook-guard` before Pi's `read`, `grep`, `find` and search-style `bash` calls. Fresh/stale graph decisions and the `GRAPHIFY_HOOK_STRICT=1` once-per-session indexed-read denial come from that same policy, so Pi matches Claude Code's hook behavior; guidance arrives as a `` steering message. It needs Pi 0.58.4 or newer: earlier versions let a steering message skip the rest of a parallel tool-call batch. The package ships inside the Graphify installation, so install Graphify persistently (`uv tool install graphifyy` or `pipx install graphifyy`, not `uvx`), restart Pi after installing or upgrading, and remove it with `graphify pi guard uninstall`. Only local file targets are inspected; the hook needs `graphify` on an absolute `PATH` entry outside the project, is bounded to a two-second, 256 KiB-in/64 KiB-out call, and fails open. The same package also loads in [Oh My Pi](https://github.com/can1357/oh-my-pi): `omp plugin install "$(graphify pi guard path)"`. The registered path contains the Python version and venv, so re-run `graphify pi guard install` after changing either, and remove the old entry with `pi remove `. + 257→ + 258→> Codex uses `$graphify` instead of `/graphify`. + 259→ + 260→
+ 261→ + 262→
+ 263→Optional extras (install only what you need) + 264→ + 265→| Extra | What it adds | Install | + 266→|---|---|---| + 267→| `pdf` | PDF extraction | `uv tool install "graphifyy[pdf]"` | + 268→| `office` | `.docx` and `.xlsx` support | `uv tool install "graphifyy[office]"` | + 269→| `google` | Google Sheets rendering | `uv tool install "graphifyy[google]"` | + 270→| `video` | Video/audio transcription (faster-whisper + yt-dlp) | `uv tool install "graphifyy[video]"` | + 271→| `mcp` | MCP stdio server | `uv tool install "graphifyy[mcp]"` | + 272→| `watch` | Auto-rebuild on file changes (`graphify watch`) | `uv tool install "graphifyy[watch]"` | + 273→| `neo4j` | Neo4j push support | `uv tool install "graphifyy[neo4j]"` | + 274→| `falkordb` | FalkorDB push support | `uv tool install "graphifyy[falkordb]"` | + 275→| `svg` | SVG graph export | `uv tool install "graphifyy[svg]"` | + 276→| `leiden` | Leiden community detection (graspologic on Python < 3.13; native backend on 3.13+) | `uv tool install "graphifyy[leiden]"` | + 277→| `ollama` | Ollama local inference | `uv tool install "graphifyy[ollama]"` | + 278→| `openai` | OpenAI / OpenAI-compatible APIs | `uv tool install "graphifyy[openai]"` | + 279→| `gemini` | Google Gemini API | `uv tool install "graphifyy[gemini]"` | + 280→| `anthropic` | Anthropic Claude API (`--backend claude`, uses `ANTHROPIC_API_KEY`) | `uv tool install "graphifyy[anthropic]"` | + 281→| `bedrock` | AWS Bedrock (uses IAM, no API key) | `uv tool install "graphifyy[bedrock]"` | + 282→| `azure` | Azure OpenAI Service (`--backend azure`, uses `AZURE_OPENAI_API_KEY` + `AZURE_OPENAI_ENDPOINT`) | `uv tool install "graphifyy[openai]"` | + 283→| `sql` | SQL schema extraction | `uv tool install "graphifyy[sql]"` | + 284→| `postgres` | Live PostgreSQL introspection (`--postgres DSN`) | `uv tool install "graphifyy[postgres]"` | + 285→| `dm` | BYOND DreamMaker `.dm`/`.dme` AST extraction (may need a C compiler + `python3-dev` if no wheel matches your platform) | `uv tool install "graphifyy[dm]"` | + 286→| `terraform` | Terraform / HCL `.tf`/`.tfvars`/`.hcl` AST extraction | `uv tool install "graphifyy[terraform]"` | + 287→| `pascal` | Pascal / Delphi `.pas`/`.dpr`/`.dpk`/`.inc` AST extraction (more accurate `calls`/`inherits` edges; falls back to a regex extractor when absent) | `uv tool install "graphifyy[pascal]"` | + 288→| `ocaml` | OCaml `.ml`/`.mli` AST extraction | `uv tool install "graphifyy[ocaml]"` | + 289→| `commonlisp` | Common Lisp `.lisp`/`.cl`/`.lsp`/`.asd` AST extraction | `uv tool install "graphifyy[commonlisp]"` | + 290→| `robot` | Robot Framework `.robot`/`.resource` extraction (suites, test cases, keywords, keyword-call and resource/library import edges) | `uv tool install "graphifyy[robot]"` | + 291→| `chinese` | Chinese query segmentation (jieba) | `uv tool install "graphifyy[chinese]"` | + 292→| `all` | Everything above | `uv tool install "graphifyy[all]"` | + 293→ + 294→
+ 295→ + 296→--- + 297→ + 298→## Make your assistant always use the graph + 299→ + 300→Run this once in your project after building a graph: + 301→ + 302→Run `graphify install` once in your project, for example `graphify claude install` or `graphify codex install` (or `graphify install --platform `). + 303→ + 304→This writes a small config file that tells your assistant to consult the knowledge graph for codebase questions, preferring scoped queries like `graphify query ""` over reading the full report or grepping raw files. + 305→ + 306→- **Hook platforms** (Claude Code, Gemini CLI): a hook fires automatically before search-style tool calls (and, on Claude Code, before reading source files one by one via the Read/Glob tools) and nudges your assistant toward the graph path. + 307→- **Instruction-file platforms** (Codex, OpenCode, Cursor, etc.): persistent instruction files (`AGENTS.md`, `.cursor/rules/`, etc.) provide the same query-first guidance. + 308→ + 309→`GRAPH_REPORT.md` is still available for broad architecture review. + 310→ + 311→Exactly which files each platform writes (CodeBuddy's `CODEBUDDY.md` + `PreToolUse` hooks, Codex's `AGENTS.md`, Kilo's native `.kilo` plugin, Cursor's `alwaysApply` rule, and so on) is covered in **[Always-on setup](https://docs.graphify.com/guides/always-on)**. + 312→ + 313→To remove graphify from all platforms at once: `graphify uninstall` (add `--purge` to also delete `graphify-out/`). Or use the per-platform command (e.g. `graphify claude uninstall`). + 314→ + 315→### CLI skills or MCP instructions? + 316→ + 317→The instructions installed by `graphify install` and those supplied with a Graphify MCP connection describe different ways to query a graph: + 318→ + 319→| Setup | Guidance to use | Query access | + 320→|---|---|---| + 321→| Local CLI and installed assistant skill | The platform instructions written by `graphify install` | Run `graphify query`, `graphify path`, and `graphify explain` against the local graph (`graphify-out/` by default). | + 322→| Connected Graphify MCP server | The connection's instructions and the tools listed by that server | Call the listed MCP tools against the graph served by that connection. Tool names and descriptions come from the server. | + 323→ + 324→Both encourage scoped graph queries before broad source searches. Choose the instructions that match the tools your assistant can actually access; neither block establishes a universal precedence rule for your assistant's instruction files. If you use both integrations, identify which graph each queries and choose a primary query path instead of copying two unconditional instruction blocks into `AGENTS.md`. + 325→ + 326→`graphify install` installs the local assistant integration; it does not establish an MCP connection. An MCP client does not need to run local CLI queries to use its connected server. A locally hosted server still needs its own Graphify installation and graph, while a hosted connection may serve a graph that is not present in the client's working directory. If the expected tools are missing, check the relevant CLI installation or MCP connection rather than treating the other setup's instructions as interchangeable. + 327→ + 328→--- + 329→ + 330→## What's in the report + 331→ + 332→`GRAPH_REPORT.md` summarizes the god nodes, communities, and key paths for broad architecture review. How the graph is built and what it contains: **[How graphify works](https://docs.graphify.com/concepts/architecture)**. + 333→ + 334→--- + 335→ + 336→## What files it handles: ~40 languages + 337→ + 338→graphify parses ~40 programming languages locally with tree-sitter AST, and maps docs, PDFs, images, and audio/video through an optional semantic pass. Full list: **[Supported inputs](https://docs.graphify.com/concepts/supported-inputs)**. + 339→ + 340→--- + 341→ + 342→## Common commands + 343→ + 344→```bash + 345→/graphify . # build graph for current folder + 346→/graphify ./docs --update # re-extract only changed files + 347→/graphify . --cluster-only # rerun clustering without re-extracting + 348→/graphify . --cluster-only --resolution 1.5 # more granular communities + 349→/graphify . --cluster-only --exclude-hubs 99 # suppress utility super-hubs from god-node rankings + 350→/graphify . --no-viz # skip the HTML, just the report + JSON + 351→/graphify . --wiki # build a markdown wiki from the graph + 352→graphify export callflow-html # Mermaid architecture/call-flow HTML (auto-regenerates on every git commit if hook is installed) + 353→ + 354→/graphify query "what connects auth to the database?" + 355→/graphify path "UserService" "DatabasePool" + 356→/graphify explain "RateLimiter" + 357→ + 358→/graphify add https://arxiv.org/abs/1706.03762 # fetch a paper and add it + 359→/graphify add # transcribe and add a video + 360→ + 361→graphify hook install # auto-rebuild on commit + branch checkout (run `graphify update .` after `git pull`: see "Recommended workflow" below) + 362→graphify merge-graphs a.json b.json # combine two graphs + 363→ + 364→graphify prs # PR dashboard: CI state, review status, worktree mapping + 365→graphify prs 42 # deep dive on PR #42 with graph impact + 366→graphify prs --triage # AI ranks your review queue (uses whatever backend is configured) + 367→graphify prs --conflicts # PRs sharing graph communities: merge-order risk + 368→``` + 369→ + 370→See the [full command reference](#full-command-reference) below. + 371→ + 372→--- + 373→ + 374→## Labeling and rendering limits + 375→ + 376→- **`--batch-size`**: 100 communities per labeling call by default, sized for a ~16k-token context (~150 tokens/community). Use `--batch-size 40` for an 8k window. Context overflow is not split-retried; failed batches keep hub-derived names. + 377→- **`--missing-only`**: `label --missing-only` keeps existing labels and names only unnamed / `Community N` entries. Use `label` (full relabel) to refresh names after the community set changes. + 378→- **`.graphify_labels.json.sig`**: records the community membership signature; labels survive a re-cluster only while it matches. + 379→- **`export html`**: 5,000-node default; above it graphify renders an aggregated community view. Use `--node-limit N`. `GRAPHIFY_VIZ_NODE_LIMIT` / `.graphifyrc` `viz_node_limit=N` apply to `cluster-only`, `update`, `watch`, and hooks, not `export html`. + 380→- **Leiden fallback**: without the `leiden` extra, clustering silently uses networkx Louvain. Switching later reshuffles every community, so plan a full relabel. + 381→ + 382→--- + 383→ + 384→## Ignoring files + 385→ + 386→graphify honors `.gitignore` and supports `--exclude` patterns and a project `.graphifyignore`. Details: **[Configuration](https://docs.graphify.com/reference/configuration)**. + 387→ + 388→ + 389→## Project configuration + 390→ + 391→Create a `.graphifyrc` in the scan root for settings that belong to the project rather than to one run. One `key=value` per line; lines beginning with `#` are comments. + 392→ + 393→**Ambiguous extensions.** Some extensions mean different languages in different projects: `.inc` is PHP on pfSense, Pascal in a Delphi tree, SQL or assembly elsewhere; `.h` is C or C++; `.m` is Objective-C or MATLAB. graphify has to pick one global default, and when it picks wrong the file does not fail: it parses as the wrong language and yields a handful of incidental nodes, so the graph looks populated while the real symbols are missing. Declare what the extension means in your repo: + 394→ + 395→``` + 396→# .graphifyrc + 397→# A language name: + 398→language.inc=php + 399→# Or an extension graphify already knows: + 400→language.tpl=.ts + 401→# Baked into the git hooks (see Team setup): + 402→viz_node_limit=0 + 403→``` + 404→ + 405→The declaration flows through classification, extractor selection, grammar and resolution, imports, `graphify watch`, and the AST cache (a file re-parsed under a different language never reuses the old entry), and the same `.graphifyrc` is read by `extract`, `update`, the hooks, the MCP server, and the `/graphify` skill. A typo is reported once on stderr and the scan continues with the defaults. How declarations interact with caching, incremental rebuilds, and import resolution: **[Configuration](https://docs.graphify.com/reference/configuration)**. + 406→ + 407→--- + 408→ + 409→## Team setup + 410→ + 411→Commit or share the graph so your whole team queries the same context, and review pull requests with graph context. See **[Share context with your team](https://docs.graphify.com/guides/team-workflows)** and **[Review pull requests](https://docs.graphify.com/guides/pull-requests)**. + 412→ + 413→--- + 414→ + 415→## Using the graph directly + 416→ + 417→Beyond your AI assistant, query `graph.json` straight from the CLI with `graphify query`, `graphify path`, and `graphify explain`. A name that matches several nodes is refused with the candidates listed rather than guessed. Scripts can read the outcome from the exit code: `0` an answer, `3` an ambiguous name, `4` no matching node. See **[Ask better graph questions](https://docs.graphify.com/guides/querying)** and the **[query reference](https://docs.graphify.com/reference/query)**. + 418→ + 419→--- + 420→ + 421→## Environment variables + 422→ + 423→Backends, API keys, hook behavior, and tuning are controlled by environment variables. Full table: **[Configuration](https://docs.graphify.com/reference/configuration)**. + 424→ + 425→- **Claude CLI model**: `GRAPHIFY_CLAUDE_CLI_MODEL` overrides the model for the `claude-cli` backend (default: Opus). `haiku` or `sonnet` are usually enough for community naming; the semantic extraction pass benefits from a larger model. + 426→ + 427→--- + 428→ + 429→## Privacy + 430→ + 431→- **Code files**: processed locally via tree-sitter. Nothing leaves your machine. A code-only corpus requires no API key: `graphify extract` runs fully offline. On a mixed repo, add `--code-only` to index just the code and skip the docs/PDFs/images that would otherwise need an LLM. + 432→- **Video / audio**: transcribed locally with faster-whisper. Nothing leaves your machine. + 433→- **Docs, PDFs, images**: sent to your AI assistant for semantic extraction (via the `/graphify` skill, using whatever model your IDE session runs). Headless `graphify extract` requires `GEMINI_API_KEY` / `GOOGLE_API_KEY` (Gemini), `MOONSHOT_API_KEY` (Kimi), `ANTHROPIC_API_KEY` (Claude), `OPENAI_API_KEY` (OpenAI), `DEEPSEEK_API_KEY` (DeepSeek), a running Ollama instance (`OLLAMA_BASE_URL`), AWS credentials via the standard provider chain (Bedrock - no API key needed, uses IAM), or the `claude` CLI binary (Claude Code - no API key needed, uses your Claude subscription). The `--dedup-llm` flag uses the same key. + 434→- **Data residency**: `graphify extract` auto-detects which provider to use based on which API key is set (priority: Gemini → Kimi → Claude → OpenAI → DeepSeek → Azure → Bedrock → Ollama). When it auto-detects, it prints the exact environment variable that chose the provider, so a run is never a surprise. To choose explicitly, pass `--backend`, or set `GRAPHIFY_BACKEND=`; set `GRAPHIFY_BACKEND=none` to disable all LLM backends (semantic extraction and the `claude -p` labeling fallback) and keep a code-only graph. For code with data-residency requirements, use `GRAPHIFY_BACKEND=ollama` / `--backend ollama` (fully local). Kimi (`MOONSHOT_API_KEY`) routes to Moonshot AI servers in China. + 435→- **No telemetry**, no usage tracking, no analytics. + 436→- **Query logging**: every `graphify query`, `graphify path`, `graphify explain`, and MCP `query_graph` call is logged to `~/.cache/graphify-queries.log` in JSON Lines format (timestamp, question, corpus, nodes returned, duration). Full subgraph responses are **not** stored by default. Set `GRAPHIFY_QUERY_LOG_DISABLE=1` to opt out, or `GRAPHIFY_QUERY_LOG=/dev/null` to silence without disabling the code path. + 437→ + 438→--- + 439→ + 440→## Troubleshooting + 441→ + 442→Common install and extraction issues and their fixes: **[Troubleshooting](https://docs.graphify.com/troubleshooting)**. + 443→ + 444→--- + 445→ + 446→## Full command reference + 447→ + 448→Every command and flag, with examples: **[CLI reference](https://docs.graphify.com/reference/overview)**. + 449→ + 450→--- + 451→ + 452→## Learn more + 453→ + 454→- [docs.graphify.com](https://docs.graphify.com): full documentation: guides, command reference, and integrations + 455→- [How it works](docs/how-it-works.md): the extraction pipeline, community detection, confidence scoring, benchmarks + 456→- [ARCHITECTURE.md](ARCHITECTURE.md): module breakdown, how to add a language + 457→- [Optional integrations](docs/docker-mcp-sqlite.md): Docker MCP Toolkit + SQLite + 458→- [The Memory Layer](https://safishamsi.gumroad.com/l/qetvlo): the book on the ideas behind graphify, the architecture end to end + 459→ + 460→--- + 461→ + 462→## Graphify Cloud + 463→ + 464→[**Graphify Cloud**](https://app.graphify.com/login) is the always-on layer built on top of graphify. Instead of a graph you rebuild on demand per folder, it keeps one live graph across your entire software development lifecycle: + 465→ + 466→- **80% fewer tokens**: always-on memory means your agents stop re-reading and re-explaining the codebase every session. + 467→- **Monorepo and cross-repo**: one connected graph across every service and repository, not a graph per folder. + 468→- **Formal verification**: check architectural and dependency invariants against the live graph. + 469→- **Code review with a bird's-eye view of your SDLC**: see how a change ripples across the whole system before you merge. + 470→- **SDLC connectors**: pull in Sentry, Jira, and more, so incidents, tickets, and code live in one graph. + 471→- **Always-on**: updates continuously in the background across code, docs, and meetings. + 472→ + 473→**[Start a free 14-day trial at app.graphify.com →](https://app.graphify.com/login)** + 474→ + 475→--- + 476→ + 477→## Contributing + 478→ + 479→Contributions are welcome. See **[CONTRIBUTING.md](CONTRIBUTING.md)** for the development setup, the test and CI-parity commands, the git workflow, and what makes a strong contribution (worked examples and extraction bug reports are the most useful). Architecture and how to add a language: [ARCHITECTURE.md](ARCHITECTURE.md). + 480→ + 481→New here? Say hi on [Discord](https://discord.gg/graphify) or in [GitHub Discussions](https://github.com/Graphify-Labs/graphify/discussions). + 482→ + 483→--- + 484→ + 485→## Contributors + 486→ + 487→ + 488→ graphify contributors + 489→ + 490→ + 491→Made with [contrib.rocks](https://contrib.rocks). + 492→ + 493→--- + 494→ + 495→## Translations + 496→ + 497→The README is available in 32 languages. Use the language switcher at the top of this file to read it in yours, or browse [`docs/translations/`](docs/translations/). To improve a translation or add a new one, open a pull request against the matching file there. + 498→ + 499→--- + 500→ + 501→## Building on graphify + 502→ + 503→Building something in the graphify ecosystem? That is encouraged. If your project uses "graphify" in its name (for example `graphify-dashboard` or `graphify-action`), please add a short note to your README clarifying that it is community-built and not affiliated with or endorsed by Graphify Labs. "graphify" and the graphify logo are marks of Graphify Labs; please do not use them in a way that implies official status. + 504→ + 505→--- + 506→ + 507→## Community and links + 508→ + 509→

+ 510→ Website + 511→ Discord + 512→ X + 513→ YouTube + 514→ Sponsor + 515→ The Memory Layer + 516→

+ 517→ \ No newline at end of file