Feat: Support pipeline-related configuration at the document level.#17212
Open
cike8899 wants to merge 2 commits into
Open
Feat: Support pipeline-related configuration at the document level.#17212cike8899 wants to merge 2 commits into
cike8899 wants to merge 2 commits into
Conversation
dosubot
Bot
added
size:XL
cike8899
added
ci
cike8899
marked this pull request as draft
July 22, 2026 03:14
cike8899
marked this pull request as ready for review
July 22, 2026 03:14
dosubot
Bot
added
the
size:XL
Contributor
📝 WalkthroughWalkthroughAdds pipeline operator configuration transforms, a Go-backend document pipeline dialog, backend-specific parser updates, and dedicated form-field components. Dataset settings and parser-change flows are rewired to use the new modules. ChangesPipeline parser configuration
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant DatasetTable
participant DocumentPipelineDialog
participant useDocumentPipelineForm
participant usePipelineOperatorNodes
participant useSetDocumentPipelineParser
DatasetTable->>DocumentPipelineDialog: render Go-backend parser dialog
DocumentPipelineDialog->>useDocumentPipelineForm: initialize and submit form
useDocumentPipelineForm->>usePipelineOperatorNodes: load pipeline operators
DocumentPipelineDialog->>useSetDocumentPipelineParser: submit transformed parser data
useSetDocumentPipelineParser-->>DatasetTable: return update result
Possibly related PRs
Suggested labels: Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
web/src/components/document-pipeline-dialog/use-document-pipeline-form.ts (1)
41-46: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winReuse
isPipelineParserConfiginstead of reimplementing the check.- const hasPipelineKeys = Object.keys(parserConfig).some((key) => - key.includes(':'), - ); - if (!hasPipelineKeys) { + if (!isPipelineParserConfig(parserConfig)) { return parserConfig; }As per coding guidelines, "search for duplicate or similar hooks and functions and reuse them instead of reimplementing the behavior inline."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/src/components/document-pipeline-dialog/use-document-pipeline-form.ts` around lines 41 - 46, Update the parser configuration check in the surrounding form logic to call the existing isPipelineParserConfig helper instead of manually checking Object.keys(parserConfig) for colon-containing keys. Preserve the current early return behavior when the configuration is not a pipeline parser configuration.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@web/src/hooks/use-document-request.ts`:
- Around line 555-557: Replace the inline queryKey array in the document-list
invalidation with a same-file DocumentKeys factory defined using as const. Reuse
the factory for this invalidation and the existing document-list invalidation,
ensuring both calls reference the shared key without raw React Query queryKey
arrays.
- Around line 508-514: Update the documentation comment describing the
Go-backend variant of useSetDocumentParser by removing the deferred-retirement
sentence about dropping the hook after the Python backend is retired, while
retaining the current endpoint contract and parallelism details.
In `@web/src/utils/pipeline-operator.ts`:
- Around line 308-342: Add a visited-node set to buildPipelineOperatorNodes and,
while following sourceToTarget from FileNodeId, stop traversal when the current
ID has already been visited. Preserve the existing ordered node collection and
mapping behavior for acyclic chains.
---
Nitpick comments:
In `@web/src/components/document-pipeline-dialog/use-document-pipeline-form.ts`:
- Around line 41-46: Update the parser configuration check in the surrounding
form logic to call the existing isPipelineParserConfig helper instead of
manually checking Object.keys(parserConfig) for colon-containing keys. Preserve
the current early return behavior when the configuration is not a pipeline
parser configuration.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 7382cf29-372d-4df6-97b5-b7af66da6ffb
📒 Files selected for processing (19)
web/src/components/builtin-pipeline-form-field.tsxweb/src/components/chunk-method-dialog/index.tsxweb/src/components/document-pipeline-dialog/index.tsxweb/src/components/document-pipeline-dialog/use-document-pipeline-form.tsweb/src/components/parse-type-form-field.tsxweb/src/components/pipeline-operator-tabs/index.tsxweb/src/components/pipeline-operator-tabs/pipeline-operator-form.tsxweb/src/hooks/parser-config-utils.tsweb/src/hooks/use-document-request.tsweb/src/hooks/use-knowledge-request.tsweb/src/hooks/use-pipeline-operator.tsweb/src/pages/dataset/dataset-setting/configuration/common-item.tsxweb/src/pages/dataset/dataset-setting/index.tsxweb/src/pages/dataset/dataset/dataset-table.tsxweb/src/pages/dataset/dataset/use-change-document-parser.tsweb/src/pages/dataset/setting/hooks.tsweb/src/pages/dataset/setting/index.tsxweb/src/pages/datasets/dataset-creating-dialog.tsxweb/src/utils/pipeline-operator.ts
💤 Files with no reviewable changes (1)
- web/src/pages/dataset/dataset-setting/configuration/common-item.tsx
Contributor
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 3
🧹 Nitpick comments (1)
web/src/components/document-pipeline-dialog/use-document-pipeline-form.ts (1)
41-46: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winReuse
isPipelineParserConfiginstead of reimplementing the check.- const hasPipelineKeys = Object.keys(parserConfig).some((key) => - key.includes(':'), - ); - if (!hasPipelineKeys) { + if (!isPipelineParserConfig(parserConfig)) { return parserConfig; }As per coding guidelines, "search for duplicate or similar hooks and functions and reuse them instead of reimplementing the behavior inline."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/src/components/document-pipeline-dialog/use-document-pipeline-form.ts` around lines 41 - 46, Update the parser configuration check in the surrounding form logic to call the existing isPipelineParserConfig helper instead of manually checking Object.keys(parserConfig) for colon-containing keys. Preserve the current early return behavior when the configuration is not a pipeline parser configuration.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@web/src/hooks/use-document-request.ts`:
- Around line 555-557: Replace the inline queryKey array in the document-list
invalidation with a same-file DocumentKeys factory defined using as const. Reuse
the factory for this invalidation and the existing document-list invalidation,
ensuring both calls reference the shared key without raw React Query queryKey
arrays.
- Around line 508-514: Update the documentation comment describing the
Go-backend variant of useSetDocumentParser by removing the deferred-retirement
sentence about dropping the hook after the Python backend is retired, while
retaining the current endpoint contract and parallelism details.
In `@web/src/utils/pipeline-operator.ts`:
- Around line 308-342: Add a visited-node set to buildPipelineOperatorNodes and,
while following sourceToTarget from FileNodeId, stop traversal when the current
ID has already been visited. Preserve the existing ordered node collection and
mapping behavior for acyclic chains.
---
Nitpick comments:
In `@web/src/components/document-pipeline-dialog/use-document-pipeline-form.ts`:
- Around line 41-46: Update the parser configuration check in the surrounding
form logic to call the existing isPipelineParserConfig helper instead of
manually checking Object.keys(parserConfig) for colon-containing keys. Preserve
the current early return behavior when the configuration is not a pipeline
parser configuration.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 7382cf29-372d-4df6-97b5-b7af66da6ffb
📒 Files selected for processing (19)
web/src/components/builtin-pipeline-form-field.tsxweb/src/components/chunk-method-dialog/index.tsxweb/src/components/document-pipeline-dialog/index.tsxweb/src/components/document-pipeline-dialog/use-document-pipeline-form.tsweb/src/components/parse-type-form-field.tsxweb/src/components/pipeline-operator-tabs/index.tsxweb/src/components/pipeline-operator-tabs/pipeline-operator-form.tsxweb/src/hooks/parser-config-utils.tsweb/src/hooks/use-document-request.tsweb/src/hooks/use-knowledge-request.tsweb/src/hooks/use-pipeline-operator.tsweb/src/pages/dataset/dataset-setting/configuration/common-item.tsxweb/src/pages/dataset/dataset-setting/index.tsxweb/src/pages/dataset/dataset/dataset-table.tsxweb/src/pages/dataset/dataset/use-change-document-parser.tsweb/src/pages/dataset/setting/hooks.tsweb/src/pages/dataset/setting/index.tsxweb/src/pages/datasets/dataset-creating-dialog.tsxweb/src/utils/pipeline-operator.ts
💤 Files with no reviewable changes (1)
- web/src/pages/dataset/dataset-setting/configuration/common-item.tsx
🛑 Comments failed to post (3)
web/src/hooks/use-document-request.ts (2)
508-514: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Remove the deferred-retirement note.
Keep the comment focused on the current Go endpoint contract; “can be dropped once the Python backend is retired” is a stale migration note.
As per coding guidelines, “Remove … ‘move later’ notes.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/src/hooks/use-document-request.ts` around lines 508 - 514, Update the documentation comment describing the Go-backend variant of useSetDocumentParser by removing the deferred-retirement sentence about dropping the hook after the Python backend is retired, while retaining the current endpoint contract and parallelism details.Source: Coding guidelines
555-557: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Use a
DocumentKeysfactory for invalidation.This adds another inline React Query key. Define a same-file
DocumentKeysfactory withas constand use it here and in the existing document-list invalidation.As per coding guidelines, “Never write raw React Query
queryKeyarrays inline.”🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/src/hooks/use-document-request.ts` around lines 555 - 557, Replace the inline queryKey array in the document-list invalidation with a same-file DocumentKeys factory defined using as const. Reuse the factory for this invalidation and the existing document-list invalidation, ensuring both calls reference the shared key without raw React Query queryKey arrays.Source: Coding guidelines
web/src/utils/pipeline-operator.ts (1)
308-342: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
No cycle guard when following the DSL edge chain.
while (currentId) { ...; currentId = sourceToTarget.get(currentId); }will loop forever ifdsl.graph.edgesever contains a cycle back to an already-visited node, hanging the tab. Add a visited-set guard.🔒️ Proposed fix
const orderedIds: string[] = []; + const visited = new Set<string>(); let currentId: string | undefined = FileNodeId; - while (currentId) { + while (currentId && !visited.has(currentId)) { + visited.add(currentId); orderedIds.push(currentId); currentId = sourceToTarget.get(currentId); }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.export function buildPipelineOperatorNodes( dsl?: DSL, pipelineParserConfig: Record<string, any> = {}, ): RAGFlowNodeType[] { if (!dsl?.graph?.nodes || !dsl?.graph?.edges) { return []; } // Build source → target map from edges (pipeline is linear) const sourceToTarget = new Map<string, string>(); for (const edge of dsl.graph.edges) { sourceToTarget.set(edge.source, edge.target); } // Follow the chain starting from File, collecting node IDs in order const orderedIds: string[] = []; const visited = new Set<string>(); let currentId: string | undefined = FileNodeId; while (currentId && !visited.has(currentId)) { visited.add(currentId); orderedIds.push(currentId); currentId = sourceToTarget.get(currentId); } // Build a lookup from node ID → node const nodeById = new Map<string, RAGFlowNodeType>(); for (const node of dsl.graph.nodes) { nodeById.set(node.id, node); } // Map ordered IDs to nodes, excluding File return orderedIds .filter((id) => id !== FileNodeId) .map((id) => nodeById.get(id)) .filter((node): node is RAGFlowNodeType => node !== undefined) .map((node) => buildOperatorNode(node, pipelineParserConfig)); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/src/utils/pipeline-operator.ts` around lines 308 - 342, Add a visited-node set to buildPipelineOperatorNodes and, while following sourceToTarget from FileNodeId, stop traversal when the current ID has already been visited. Preserve the existing ordered node collection and mapping behavior for acyclic chains.
cike8899
marked this pull request as draft
July 22, 2026 03:45
cike8899
marked this pull request as ready for review
July 22, 2026 03:45
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Feat: Support pipeline-related configuration at the document level.