[airborne12]

What problem does this PR solve?

Related PR: #69292

Problem Summary: Backport #69292 to branch-4.2. It contains the commit of #69560, the backport of #69399, which it needs: binding runs before CTE inlining, which copies scans with virtual columns. Merge #69560 first.

score() only worked as ORDER BY score() LIMIT n directly above the scan whose MATCH defines it. A minimum score filter without that TopN evaluated the unbound score() placeholder as zero and silently dropped every row:

SELECT id FROM documents WHERE content MATCH 'apple' AND score() > 0.00001;  -- returned nothing

BindScoreToOlapScan, which replaces CheckScoreUsage, binds score() to the scan of its SELECT at the start of rewrite and rejects the uses it cannot bind. The minimum score stays a predicate of the scan, and PushDownScoreTopNIntoOlapScan still pushes a TopN on the score together with the minimum. In the BE, an expression on the score reads the score column instead of evaluating Score(), and MATCH projections do not add to the score. See #69292 for the supported and rejected uses.

Branch adaptations

• The SegmentIterator of the branch collects the columns of pushed-down expressions in _extract_common_expr_columns instead of per-column states, so the score column is marked there, and the new BE test checks _common_expr_columns and _is_common_expr_column. The branch reads the columns collected there before it evaluates the expressions, also virtual ones, and a virtual slot uses a column that is already read.
• When limit + offset overflows, PushDownScoreTopNIntoOlapScan keeps the behavior of the branch and skips the push-down instead of failing.
• SearchDslParser is imported from its package on the branch.
• The branch does not have [fix](cache) Share cache eligibility for nondeterministic expressions #68672, which keeps predicates on virtual columns out of the condition cache. The minimum score is now a predicate of the scan, and a score depends on every rowset, so a cached result goes stale when a rowset is added: without the change, test_score_filter_matrix lost the rows of the first rowset. VirtualSlotRef::get_digest returns 0 as on master, the part of [fix](cache) Share cache eligibility for nondeterministic expressions #68672 that this needs, with the VirtualSlotRef tests of [fix](cache) Share cache eligibility for nondeterministic expressions #68672 that the branch can run.

Release note

score() works in the select list, window functions and ORDER BY of a SELECT on one table whose WHERE clause has MATCH or SEARCH, with an optional minimum score score() > x or score() >= x, without ORDER BY score() LIMIT. A scored SELECT in a subquery, view, CTE or set operation still needs ORDER BY score() LIMIT. Other uses of score() are rejected instead of evaluating it as 0.

Check List (For Author)

• Test
  • Regression test
  • Unit Test
  • Manual test
• Behavior changed:
  • Yes. A minimum score works without ORDER BY score() LIMIT, and unsupported uses of score() fail planning instead of evaluating it as 0. A minimum score without a TopN needs the BE change, so BEs must be upgraded before FEs.
• Does this need documentation?
  • Yes. The score() documentation should describe where score() can be used.

Validation

Validated together with the other two backports (#69399, #69419 and #69292 on branch-4.2 at ecbc5fe), in a local ASAN build:

• PASS: FE unit tests LogicalPlanDeepCopierTest, CTEInlineTest, InferPredicateByReplaceTest, InferPredicatesTest, UnequalPredicateInferTest, BindScoreToOlapScanTest, SearchExpressionTest, TopNRuntimeFilterTest, PullUpProjectExprUnderTopNTest and MaterializedViewUtilsTest (280 tests), and FE Checkstyle.
• PASS: ./build.sh --fe --be.
• PASS: BE unit tests SegmentIteratorScoreCommonExprTest, SegmentIteratorApplyIndexExprTest and VirtualSlotRefTest (31 tests); clang-format 16 on the changed BE files.
• PASS: the 23 regression suites of the branch that use score(), including test_cte_inline_virtual_column, infer_match_predicate, test_score_filter_usage and test_score_filter_matrix, with unchanged .out files.

🤖 Generated with Claude Code

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[airborne12]

run buildall

[hello-stephen]

FE UT Coverage Report

Increment line coverage 82.42% (211/256) 🎉
Increment coverage report
Complete coverage report

[hello-stephen]

FE Regression Coverage Report

Increment line coverage 83.20% (213/256) 🎉
Increment coverage report
Complete coverage report

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 100.00% (36/36) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	76.41% (34528/45186)
Line Coverage	61.24% (390023/636846)
Region Coverage	58.08% (328742/566062)
Branch Coverage	59.02% (151678/257005)