Dinky Roadmap

[aiwenmo]

The following list is the latest roadmap.

This list shows the main feature planning of the project in the future. You are welcome to supplement, design, discuss and code specific new features.

以下列表是最新的 Roadmap（保持最新）。
该列表展示了项目未来的主要特性规划，欢迎你来对具体的新特性进行补充、设计、讨论和编码。

---

Roadmap

• Refine data development: Optimize user prompts and interactions, and add various auxiliary functions to enhance the development and debugging capabilities of streaming jobs.
• Optimize real-time lake warehouse: Explore the ability of real-time data entry into the lakehouse, self optimization of lake warehouse, and self optimization of data operations to ensure stable and efficient data architecture.
• Multi version governance components: There are numerous incompatible versions in the components of big data, making it difficult for users to build data architectures. Dinky will explore multi version support and compatibility solutions for numerous components.
• Implementing a high availability architecture: Design and implement a high availability architecture to support smooth and seamless development and monitoring of operations in the face of a large number of personnel, ensuring the security of developing and managing all operations within the enterprise.
• Support more middleware: integrate more computing engines and storage systems, establish task and metadata standards, and provide unified docking and service methods.
• Support big model assistance: Using big models to assist can improve work efficiency. Dinky will introduce big models in the future to assist data development, anomaly resolution, and other work.

打磨数据开发：优化用户提示与交互，并添加各种辅助功能以增强流作业的开发和调试能力。
优化实时湖仓：探索在数据实时入湖、湖仓自优化、数据作业自优化的能力，保证数据架构稳定高效。
治理组件多版本：大数据的组件中版本存在众多不兼容，使用户在构建数据架构时困难重重，Dinky 将探索众多组件的多版本支持与兼容方案。
实现高可用架构：设计并实现高可用架构，以支持在面对大量人员开发及大量作业监控运维时的平稳流畅，使其具备开发及管理企业内所有作业的保障。
支持更多中间件：集成更多计算引擎及存储系统，制定任务及元数据标准，提供统一的对接及服务方式。
支持大模型辅助：通过大模型来辅助可以提升工作效率，Dinky 未来将引入大模型，辅助数据开发、异常解决等工作。

Dinky 1.3.0

• Integrate Apache Amoro for managing data lakes
• Mock test support flink application mode
• Support catch the schema change log of FlinkCDC Pipeline for tracing back
• ...

集成 Apache Amoro，用于治理数据湖
Mock 测试支持 Flink Application 模式
支持捕获 FlinkCDC Pipeline 的结构变化日志用于追溯
...

[codegit10001]

支持推送sql任务到dolphinscheduler，支持读取dolphinscheduler里面相关sql任务到dinky，实现dinky前端写sql任务，dolphinscheduler进行调度执行。

[leeoo]

期望注册中心能够支持添加 Paimon 数据源