The Databricks AI Development Suite enhances AI-driven development capabilities by providing your programming assistants (Claude Code, Cursor, etc.) with reliable Databricks knowledge and best practices.
It includes a Python library, an MCP server equipped with more than 50 tools, a Markdown skill library that explains the Databricks development model, and a web version of the building application. You can build Spark data pipelines, job tasks, data kanbans, knowledge assistants, and deploy machine learning models more efficiently and intelligently.
The advantage is that your AI programming assistant can directly call Databricks ‘functions and development models, allowing you to use built-in management specifications and best practices to develop data and artificial intelligence applications more efficiently.
If you think of today’s AI development environment as an evolving chain, then AI Dev Kit What we are doing is actually not to recreate a tool, but to fill the gap between the “AI programming assistant” and the real data platform.
The most direct starting point for this project is not to teach people how to write models and adjust parameters, but to answer a more realistic question: how to make these AI assistants truly understand and operate data and AI platforms like Databricks when developers are already writing code with tools like Claude Code or Cursor, rather than just staying at the level of generating code.
As a result, the structure of this development kit became very interesting. On one hand, it provides a Python library and a set of MCP servers to encapsulate Databricks ‘capabilities into “tools” that can be called by AI; on the other hand, it also uses a complete set of skills libraries in the form of Markdown to write Databricks’ development models, usage methods, and best practices into “knowledge” that AI can understand. In other words, it not only gives the AI hand, but also gives the AI brain.
When these capabilities are combined, the role of the AI programming assistant changes. Instead of just generating a piece of Spark code, it can act around the entire development process: understanding data, building pipelines, organizing tasks, and even participating in model deployment. This change is not achieved by adding complex functions, but by allowing AI to directly “connect” to the way Databricks works.
There is a key point here: the project does not focus on “stronger models” but on “more reliable ways to use them.” With these built-in tools and skills, AI follows existing development specifications when operating Databricks, rather than guessing from zero every time. This makes the entire process closer to real engineering rather than a one-time experimental script.
You will gradually realize that this project is actually doing something lower-level-it is turning Databricks into an environment that can be used natively by AI. Developers no longer just call APIs, but use platform capabilities indirectly through AI agents; and AI no longer just answers questions, but participates in the actual development process.
Therefore, this is not so much an “AI application development tool” as it is more like a connection layer: connecting the AI programming assistant while connecting the data and the machine learning platform. After this connection is established, building data pipelines, job tasks, data kanban, and even deployment models is no longer a decentralized step, but a process that can be completed continuously in the same context.
This change does not bring about an improvement in a certain function, but a change in development methods. When AI can understand the platform, invoke tools, and follow established patterns, development efficiency will naturally increase, but more importantly, the entire process begins to become controllable, reusable, and closer to the real production environment.
This also explains why the project doesn’t look like a “tutorial” or “example” in the traditional sense. Instead of trying to teach you what to do step by step, it simply gives you a well-organized way for AI and Databricks to work together. Under this premise, developers have less things to do: no longer build from scratch, but advance within existing structures.
From this perspective, what the Databricks AI development kit expresses is actually a very clear direction-future development is not just the relationship between people and code, nor the relationship between people and AI, but AI as an executor, directly participate in the real system. And this project provides a starting point for this relationship to be implemented.
Github：https://github.com/databricks-solutions/ai-dev-kit
Oil tubing: