[feature](function) Impl the function map_concat like spark #59130

nagisa-kunhah · 2025-12-17T18:59:48Z

Fix #51706

What problem does this PR solve?

Issue Number: close #xxx

Fix issue: #51706

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes. [Feature](docs) Impl the function map_concat like spark doris-website#3220

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-12-17T18:59:53Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

zclllyybb · 2026-01-15T12:16:26Z

fe/fe-core/src/main/java/org/apache/doris/nereids/util/TypeCoercionUtils.java

            return processCreateMap((CreateMap) boundFunction);
        }

-        // type coercion


dont modify this

zclllyybb · 2026-01-15T12:17:03Z

...ore/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/MapConcat.java

+     * Compute signatures when arity > 0.
+     * Extract key and value types, find common type, and construct signatures.
+     */
+    private List<FunctionSignature> computeNonEmptySignatures() {


没必要再单独抽一个函数出来了。

nagisa-kunhah · 2026-01-17T08:37:38Z

run buildall

nagisa-kunhah · 2026-01-17T09:34:21Z

run buildall

doris-robot · 2026-01-17T10:25:30Z

TPC-H: Total hot run time: 32526 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c35513dc7cf7c3f9f9f4a369f632740eb3bb8f7d, data reload: false

------ Round 1 ----------------------------------
q1	17659	4242	4049	4049
q2	2065	355	278	278
q3	10101	1307	738	738
q4	10212	849	315	315
q5	7511	2066	1994	1994
q6	200	174	143	143
q7	929	829	643	643
q8	9273	1432	1246	1246
q9	5050	4613	4680	4613
q10	6812	1814	1402	1402
q11	560	310	283	283
q12	738	766	583	583
q13	17768	3883	3153	3153
q14	312	290	277	277
q15	584	517	515	515
q16	687	687	625	625
q17	672	785	531	531
q18	6740	6463	6912	6463
q19	1445	1035	712	712
q20	435	392	288	288
q21	3431	2777	2573	2573
q22	1172	1164	1102	1102
Total cold run time: 104356 ms
Total hot run time: 32526 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4334	4223	4245	4223
q2	358	443	327	327
q3	2241	2791	2446	2446
q4	1519	1914	1631	1631
q5	4534	4315	4274	4274
q6	229	172	129	129
q7	2015	1984	1855	1855
q8	2558	2438	2429	2429
q9	7045	7066	7481	7066
q10	2439	2753	2116	2116
q11	552	465	434	434
q12	658	688	575	575
q13	3407	3895	3169	3169
q14	278	282	255	255
q15	526	498	477	477
q16	633	651	614	614
q17	1106	1252	1321	1252
q18	7389	7344	7248	7248
q19	844	810	801	801
q20	1875	1961	1854	1854
q21	4595	4323	4104	4104
q22	1088	1034	950	950
Total cold run time: 50223 ms
Total hot run time: 48229 ms

doris-robot · 2026-01-17T10:36:18Z

TPC-DS: Total hot run time: 174162 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c35513dc7cf7c3f9f9f4a369f632740eb3bb8f7d, data reload: false

query5	4440	639	478	478
query6	326	230	202	202
query7	4219	468	267	267
query8	333	241	233	233
query9	8753	2901	2897	2897
query10	514	395	328	328
query11	15221	15093	14864	14864
query12	177	113	111	111
query13	1236	492	377	377
query14	6018	3127	2866	2866
query14_1	2700	2709	2694	2694
query15	194	188	169	169
query16	991	486	475	475
query17	1090	688	576	576
query18	2472	436	339	339
query19	224	222	198	198
query20	121	119	119	119
query21	229	141	119	119
query22	4153	3870	4052	3870
query23	16002	15556	15214	15214
query23_1	15390	15568	15451	15451
query24	7106	1560	1180	1180
query24_1	1191	1195	1184	1184
query25	547	457	402	402
query26	1244	271	153	153
query27	2774	445	286	286
query28	4590	2187	2166	2166
query29	768	544	446	446
query30	318	247	210	210
query31	786	635	563	563
query32	86	77	75	75
query33	535	375	321	321
query34	918	880	536	536
query35	718	778	684	684
query36	883	910	802	802
query37	143	97	89	89
query38	2765	2787	2760	2760
query39	773	746	721	721
query39_1	718	720	716	716
query40	228	143	124	124
query41	73	70	67	67
query42	110	105	107	105
query43	492	437	426	426
query44	1374	766	760	760
query45	187	185	178	178
query46	866	963	595	595
query47	1364	1502	1359	1359
query48	324	336	245	245
query49	622	439	375	375
query50	638	284	218	218
query51	3795	3793	3798	3793
query52	107	109	100	100
query53	294	340	275	275
query54	303	287	271	271
query55	88	89	83	83
query56	330	317	317	317
query57	1024	995	890	890
query58	282	265	270	265
query59	2158	2143	2110	2110
query60	342	350	327	327
query61	214	145	143	143
query62	386	371	309	309
query63	306	267	268	267
query64	4881	1233	962	962
query65	3859	3698	3806	3698
query66	1450	419	311	311
query67	15579	15546	15454	15454
query68	2508	1106	782	782
query69	459	373	337	337
query70	975	943	996	943
query71	343	325	297	297
query72	5287	3205	3196	3196
query73	618	737	326	326
query74	8726	8744	8570	8570
query75	2746	2829	2480	2480
query76	2274	1066	698	698
query77	357	390	307	307
query78	9803	9828	9074	9074
query79	2356	923	591	591
query80	1763	573	494	494
query81	552	268	237	237
query82	1015	149	116	116
query83	361	263	238	238
query84	246	117	97	97
query85	875	486	423	423
query86	403	327	285	285
query87	2974	2857	2766	2766
query88	3506	2599	2567	2567
query89	396	362	325	325
query90	1965	182	178	178
query91	170	162	135	135
query92	78	75	68	68
query93	1170	962	543	543
query94	636	306	280	280
query95	600	405	310	310
query96	636	526	236	236
query97	2346	2338	2312	2312
query98	216	201	202	201
query99	602	575	493	493
Total cold run time: 249574 ms
Total hot run time: 174162 ms

doris-robot · 2026-01-17T10:41:21Z

ClickBench: Total hot run time: 26.8 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c35513dc7cf7c3f9f9f4a369f632740eb3bb8f7d, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.04	0.04
query3	0.25	0.09	0.08
query4	1.60	0.11	0.11
query5	0.27	0.25	0.25
query6	1.14	0.65	0.66
query7	0.03	0.02	0.02
query8	0.05	0.04	0.05
query9	0.58	0.50	0.49
query10	0.55	0.55	0.55
query11	0.15	0.10	0.11
query12	0.16	0.10	0.11
query13	0.60	0.59	0.59
query14	0.95	0.96	0.94
query15	0.80	0.78	0.78
query16	0.40	0.41	0.37
query17	1.03	1.03	1.06
query18	0.24	0.23	0.22
query19	1.88	1.82	1.87
query20	0.02	0.02	0.01
query21	15.44	0.29	0.15
query22	5.13	0.05	0.05
query23	15.83	0.29	0.10
query24	2.30	0.35	0.66
query25	0.08	0.11	0.06
query26	0.14	0.12	0.12
query27	0.06	0.05	0.06
query28	4.35	1.06	0.88
query29	12.54	3.91	3.12
query30	0.28	0.14	0.13
query31	2.82	0.65	0.40
query32	3.23	0.56	0.46
query33	3.06	3.03	3.02
query34	16.32	5.00	4.40
query35	4.41	4.49	4.43
query36	0.64	0.50	0.50
query37	0.11	0.06	0.07
query38	0.07	0.04	0.03
query39	0.05	0.03	0.03
query40	0.16	0.13	0.14
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 98.05 s
Total hot run time: 26.8 s

zclllyybb

LGTM

github-actions · 2026-01-17T20:25:53Z

PR approved by at least one committer and no changes requested.

zclllyybb · 2026-01-17T20:26:28Z

your new added BE-UT case failed. please check them.

nagisa-kunhah · 2026-01-18T15:12:34Z

run buildall

hello-stephen · 2026-01-18T16:21:23Z

FE UT Coverage Report

Increment line coverage 2.70% (1/37) 🎉
Increment coverage report
Complete coverage report

nagisa-kunhah · 2026-01-26T16:42:46Z

run buildall

hello-stephen · 2026-01-26T17:49:49Z

FE UT Coverage Report

Increment line coverage 2.70% (1/37) 🎉
Increment coverage report
Complete coverage report

doris-robot · 2026-01-26T18:18:43Z

TPC-H: Total hot run time: 32800 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9dd20deb58ca39def26f44c685e0543aa1b955c4, data reload: false

------ Round 1 ----------------------------------
q1	17629	5310	5070	5070
q2	2031	320	236	236
q3	10153	1315	767	767
q4	10198	820	316	316
q5	7505	2165	1931	1931
q6	201	185	150	150
q7	892	732	608	608
q8	9263	1437	1116	1116
q9	5461	4854	4782	4782
q10	6864	1965	1555	1555
q11	507	294	286	286
q12	395	382	224	224
q13	17787	4052	3191	3191
q14	234	241	217	217
q15	913	839	823	823
q16	669	677	620	620
q17	644	886	422	422
q18	6763	6505	7552	6505
q19	1736	1049	645	645
q20	408	376	235	235
q21	2970	2322	2068	2068
q22	1215	1127	1033	1033
Total cold run time: 104438 ms
Total hot run time: 32800 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5665	5541	5435	5435
q2	263	350	270	270
q3	2381	2845	2568	2568
q4	1540	1929	1471	1471
q5	4793	4516	4432	4432
q6	228	189	138	138
q7	2021	1954	1915	1915
q8	2559	2359	2395	2359
q9	7595	7654	7449	7449
q10	2822	2956	2546	2546
q11	534	448	430	430
q12	611	725	558	558
q13	3542	3998	3262	3262
q14	270	278	264	264
q15	831	799	794	794
q16	633	672	627	627
q17	1068	1335	1367	1335
q18	7391	7323	7331	7323
q19	826	786	805	786
q20	1978	2066	1886	1886
q21	4525	4217	4083	4083
q22	1057	1072	990	990
Total cold run time: 53133 ms
Total hot run time: 50921 ms

doris-robot · 2026-01-26T18:35:30Z

ClickBench: Total hot run time: 28.24 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9dd20deb58ca39def26f44c685e0543aa1b955c4, data reload: false

query1	0.05	0.04	0.05
query2	0.10	0.05	0.05
query3	0.26	0.09	0.09
query4	1.61	0.11	0.11
query5	0.27	0.24	0.25
query6	1.18	0.69	0.67
query7	0.04	0.03	0.02
query8	0.05	0.04	0.04
query9	0.56	0.51	0.50
query10	0.54	0.53	0.53
query11	0.14	0.09	0.10
query12	0.14	0.10	0.10
query13	0.63	0.61	0.61
query14	1.06	1.06	1.04
query15	0.88	0.87	0.86
query16	0.41	0.40	0.39
query17	1.07	1.12	1.09
query18	0.22	0.22	0.20
query19	2.14	2.00	1.94
query20	0.02	0.01	0.02
query21	15.42	0.24	0.14
query22	5.16	0.05	0.05
query23	15.87	0.28	0.11
query24	1.01	0.74	0.31
query25	0.07	0.09	0.06
query26	0.13	0.13	0.14
query27	0.07	0.05	0.07
query28	3.55	1.16	0.96
query29	12.60	3.90	3.17
query30	0.28	0.13	0.12
query31	2.83	0.62	0.40
query32	3.24	0.60	0.50
query33	3.19	3.28	3.21
query34	16.10	5.37	4.75
query35	4.81	4.78	4.78
query36	0.66	0.51	0.50
query37	0.13	0.07	0.07
query38	0.07	0.05	0.04
query39	0.04	0.03	0.03
query40	0.19	0.17	0.16
query41	0.09	0.04	0.03
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 96.96 s
Total hot run time: 28.24 s

doris-robot · 2026-01-26T19:51:04Z

BE UT Coverage Report

Increment line coverage 76.92% (50/65) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.48% (19146/36485)
Line Coverage	35.87% (177996/496209)
Region Coverage	32.32% (137496/425483)
Branch Coverage	33.26% (59535/178982)

hello-stephen · 2026-01-26T20:52:40Z

BE Regression && UT Coverage Report

Increment line coverage 86.15% (56/65) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	73.18% (26170/35762)
Line Coverage	56.22% (278672/495665)
Region Coverage	53.86% (231608/429988)
Branch Coverage	55.63% (100001/179750)

hello-stephen · 2026-01-26T21:03:07Z

FE Regression Coverage Report

Increment line coverage 81.08% (30/37) 🎉
Increment coverage report
Complete coverage report

nagisa-kunhah added 25 commits December 15, 2025 02:13

feat: map_concat

106afda

fix: optimize code

eaddffe

fix: nullable

6473a03

feat: FunctionMapConcatTest

930ef85

fix: ColumnNullable

7b0bf75

feat: function_map_concat_test

f0970be

fix: function_map_concat_test

cef3520

remove: FunctionMapConcatTest.TestComplexTypes

e5c3bd8

fix: get_key_value_type

7869361

fix: TestComplexTypes of array

aea2e7f

fix: check_function_all_arg_comb

71a6cb7

feat: Test with double, float and decimalv2

d42447c

feat: Test single map

fc221f6

feat: extend map_concat to handle struct-typed values

053760a

fix: compile error

3c585c6

feat: test_map_concat

174cab1

fix: handle null values in arguments to prevent crash

65de5e5

fix: test_map_concat.out

d428f86

test: expand test coverage for mapconcat function

41c7f69

fix: style issue

4332c22

fix: stype issue

94bd981

fix: style issue

7bfec0f

fix: style issue

958eac9

fix: crash when “select map_concat();”

fb58202

fix: optimize test

109c9af

nagisa-kunhah requested a review from zclllyybb as a code owner December 17, 2025 18:59

zclllyybb self-assigned this Dec 17, 2025

fix: license issue

41ed41f

nagisa-kunhah changed the title ~~[feature] Impl the function map_concat like spark #51706~~ [feature](function) Impl the function map_concat like spark #51706 Dec 18, 2025

zclllyybb reviewed Jan 15, 2026

View reviewed changes

nagisa-kunhah added 2 commits January 17, 2026 01:55

fix: comment

be768a6

refactor

88045c4

nagisa-kunhah requested a review from zclllyybb January 17, 2026 08:00

nagisa-kunhah added 2 commits January 17, 2026 17:29

fix: comment

5c33069

fix: comment

c35513d

zclllyybb previously approved these changes Jan 17, 2026

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 17, 2026

nagisa-kunhah and others added 2 commits January 18, 2026 20:31

Merge branch 'master' into feat_map_concat

42829ff

fix: ut

49bfdd6

nagisa-kunhah dismissed zclllyybb’s stale review via 49bfdd6 January 18, 2026 15:09

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jan 18, 2026

Merge branch 'master' into feat_map_concat

9dd20de

[feature](function) Impl the function map_concat like spark #59130

Are you sure you want to change the base?

[feature](function) Impl the function map_concat like spark #59130

Uh oh!

Conversation

nagisa-kunhah commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Dec 17, 2025

Uh oh!

zclllyybb Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

zclllyybb Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

nagisa-kunhah commented Jan 17, 2026

Uh oh!

nagisa-kunhah commented Jan 17, 2026

Uh oh!

doris-robot commented Jan 17, 2026

Uh oh!

doris-robot commented Jan 17, 2026

Uh oh!

doris-robot commented Jan 17, 2026

Uh oh!

zclllyybb left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 17, 2026

Uh oh!

zclllyybb commented Jan 17, 2026

Uh oh!

nagisa-kunhah commented Jan 18, 2026

Uh oh!

hello-stephen commented Jan 18, 2026

FE UT Coverage Report

Uh oh!

nagisa-kunhah commented Jan 26, 2026

Uh oh!

hello-stephen commented Jan 26, 2026

FE UT Coverage Report

Uh oh!

doris-robot commented Jan 26, 2026

Uh oh!

doris-robot commented Jan 26, 2026

Uh oh!

doris-robot commented Jan 26, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Jan 26, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Jan 26, 2026

FE Regression Coverage Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

nagisa-kunhah commented Dec 17, 2025 •

edited

Loading