NOTE: This was my final project for the Artificial Intelligence course at UNIFESP  

Tutorial: Tool Calling and Model Context Protocol (MCP) for Java Applications Using Local LLM

1. Abstract

Business Applications integration with Large Language Models (LLM) is proven to be helpful in multiple areas. The programming language Java is among the most used programming languages for Business Application but is not known for its strong adoption by Data Scientists, however, using Java application can make use of Tool Calling for integration with LLM. In this tutorial we share how a Java application can be tuned for high performance and integrated with local LLM using Tool Calling and Model Context Protocol (MCP).

2. Introduction

Business Applications integration with Large Language Models (LLM) provide better knowledge management, customer operations, improvements in marketing and sales, analytics and reporting, automation and in many other areas (Arslan et all, 2026). Generative AI at work has also proven to increase users' performance (Brynjolfsson at all, 2025). 

Large Language Models primarily generate only text, and have no connection with external systems, Tool Calling is a solution for an intermediate software to be able to consume response from LLM, interpret it and call internal code which response is fed into the LLM context. This can be achieved by exposing the LLM to known programming language concepts during pre-training and tool calling tokens during post-training, this way the LLM will generate tokens when it wants to invoke external systems and these tokens can be interpreted by the software running the LLM (Schick et all, 2023).

Figure 1: Pre-training phase of model to make it capable of tool calling

Tool Calling implies that the software running the LLM may have tools written in the same programming language. Model Context Protocol establishes a way to standardize the call to external tools that may be local or remote, and it is not tight to the technology used to connect to the LLM (Linux Foundation, 2026).

Figure 2: MCP diagram

The programming language Java is among the most used programming languages for Business Application due its performance and stability, variety of mature libraries and frameworks, portability and scalability (Salo, 2026). However, Java is not known for its strong adoption by Data Scientists and its frameworks for this area are not widely adopted, such as Pytorch with Python (Li, 2026). The Java application acts as the orchestrator of Middleware software and connect to LLMs using Tool Calling and MCP with frameworks such as Langchain4J, which can run as part of Quarkus  (Quarkiverse, 2026), which offers other libraries and native compilation for maximum performance and cloud-native binaries that requires no Java Virtual Machine to run (Quarkus, 2026), making Quakus the faster Java framework when using in native mode (Cummins, 2026). 

Figure 3: Tool Calling diagram

The rise of AI usage in companies comes with the concern about privacy and costs. A small fine-tuned LLM may be used in order to keep the data in the company network, decrease costs and have specialized small models on company private data. (Jerrom, 2025). A LLM even when small, can still have high computation costs for inference and fine-tuning, hence for local models it is important to select a quantization, which accelerate inference and training (Gonga et all, 2025).The open model Qwen 3.6 35BA3B can run its quantized version on small processing languages, be fine-tuned and it “excels in tool calling” (Unsloth, 2026).

This Tutorial introduces to the reader to Tool Calling, MCP, local models and Java development with Quarkus.

3. Objectives

• Introduce the theoretical concepts related to Tool Calling and Model Context Protocol (MCP)

• Guide the user to setup a server to run a local Large Language Model (LLM)

• Guide the user to setup and develop a simple Java application that makes use of Tool Calling

• Demonstrate the use of MCP using a Java application

• Showcase a real world case of MCP and Tool Calling using Java

4. Methodology

This tutorial initially introduces the concepts and the relevancy of learning Tool Calling and MCP with Java and Local LLMs. The hands-on guide starts by preparing the environment for the code tasks, then the actual coding and finally a section for tuning the application for better performance. The Tutorial last part is to share with the user a real world example of Tool Calling, MCP and Java to create a TODO Application. 

5. Tutorial Process

This tutorial is divided in the following parts: setup local LLM server; Setup the Java application; the development of a tool; MCP setup; Performance Tuning with Native Compilation; Show case: Image to Database.

This tutorial was run on a Fedora 43 operating system running on a Lenovo P16V laptop with 64GB of RAM.

5.1 Local LLM Setup

A local LLM local execution requires an application which is capable of understanding the LLM architecture, read the model weights and make the LLM available via HTTP using an OpenAI compatible REST API. Popular choices are listed below:

• ollama: Ollama is by far the most popular choice because it hides the complexity from the user of downloading and managing the model parameters, but it does not allow the user to configure low level details such as use of GPU versus CPU. It also lacks the community and quick updates as other tools do. Available in https://ollama.com;

• vllm: vllm has fast API updates and a big community, and it is more suitable for multi users serving. Available in https://vllm.ai;

• llama.cpp: llama.cpp has multiple distributions for multiple architectures, offers different tools for dealing with different tasks for LLMs (quantization, benchmark and so on) and has a server that can be fine-tuned for the target hardware where it is running. This is the tool that selected for this Tutorial and it is available in https://llama-cpp.com;

Steps to setup and run a model using llama.cpp:

1. Download llama.cpp: One must select the distribution of llama.cpp that is compatible with their system, for this tutorial hardware and operating system the distribution Ubuntu x64 (Vulkan) was selected. llama.cpp can be downloaded from the following link: https://github.com/ggml-org/llama.cpp/releases/latest

2. Model Selection: Select a model and a quantization to run locally. In this tutorial hardware the model Qwen3.6-35B-A3B-UD-Q4_K_M with Multiple Token Prediction (MTP) support and quantized by unsloth was the selected one. One can learn more about Qwen 3.6 35A3B and its quantized versions on unsloth page accessible at https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF

3. Run the server: Descompress the llama.cpp download file and find the binary file llama-server. Using a terminal, run the file with the parameters to set up the server execution. Make it sure that the parameter ‘-m’ points to the downloaded model file. For this tutorial the following parameters were used:

./llama-server -m /opt/models/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf     --ctx-size 16384     --temp 0.6     --top-p 0.95     --top-k 20     --min-p 0.00 --reasoning off -t 8 --mlock -ngl 0 -b 1024      --spec-type draft-mtp --spec-draft-n-max 3

4. Test the installation: Once the server is started, users can chat with the LLM using a web browser and accessing the address http://localhost:8080. The displayed home page screenshot is in Figure 4

Figure 4: Llama.cpp server home page

One must keep the llama.cpp server running for this tutorial since the Java application connects to it.

5.2 Java application Setup

The Java application that will contain the tool requires at least a Java JDK distribution installation for the development. For this tutorial an external Maven and VSCode are used, but one should be able to follow along only with Java SDK.

Steps to setup and run the Java application:

1. Download and install Java: The selected Java version is Java JDK 26 with the distribution OpenJDK, but other distributions should also work for this. The Java JDK can be downloaded on https://jdk.java.net/26 . Linux users can make use of sdkman, which is available on https://sdkman.io. Once sdk is installed, Java JDK can be installed using the following command on a terminal: sdk install java 26.0.1-open. A Java installation can be verified using the command java -version  as seen in Figure 5.

Figure 5: Java installation verification

2. Generate the application skeleton: A Quarkus application can be generated using the started web application which is available on code.quarkus.io. The Quarkus platform version for this tutorial is 3.37 and the application name is optional, but the extensions must be selected as displayed in Figure 6. The list of extensions is below. 

1. LangChain4j OpenAI: Provides the integration what a LLM Server

2. LangChain4j Model Context Protocol client: Provides the integration with existing MCP Servers

3. LangChain4j Model Context Protocol Server: Provides the ability to expose the application tools using a MCP Server

4. Quarkus REST: Provides access to the dev panel that is useful for sending messages to the LLM API using the web browser.

            Once the extensions are selected, click on the Generate your application and uncompress the downloaded file.

 Figure 6: List of extensions used on this tutorial

3. Test the Java application skeleton: Using a terminal, go into the application directory, modify the file src/main/resource/application.properties and add the port configuration quarkus.http.port=8180 to change the application port and avoid conflicts. From the application root use the command ./mvnw quarkus:dev to start the application. The first run will download all dependencies, once it is done the terminal will show the Quarkus brand as seen in Figure 7  and the address to access the application, use it with a web browser to see Quarkus home page as seen in Figure 8. It is important to make it clear that the application should be running for the next steps of this Tutorial.

Figure 7: Quarkus application has started

Figure 8: Quarkus application default Home Page

4. Configure the LLM Integration: The application is ready to be integrated to any LLM provider that is Open AI compatible. To configure the application to connect to the llama.cpp server from section 5.1 one must use the property quarkus.langchain4j.openai.base-url with value http://localhost:8080/v1 and a Java class to register the AI Service. The steps

1. Open the file src/main/resources/application.properties and add the line quarkus.langchain4j.openai.base-url=http://localhost:8080/v1/. The resulting file is displayed in Figure 9

2. Create a Java file AiService.java in directory src/main/java/org/acme/ (all classes for this tutorial will be placed on this directory, therefore a package declaration is required) and declare an interface that has the annotation @io.quarkiverse.langchain4j.RegisterAiService.The file with the full content is displayed in Figure 10

Figure 9: Application configuration in application.properties

Figure 10: Code for registering the AI Service

5. Chat with the LLM: Access the dev UI and send a message to the LLM to ensure the connection is working properly. The Dev UI is accessible from the following URL http://localhost:8180/q/dev-ui/quarkus-langchain4j-core/chat. The chat to the LLM from the Java application is illustrated on Figure 11;

Figure 11: Chat with the local LLM from the Java application

5.3 Tool development

The step 5.2 finished with a chat with a LLM, but it is not invoking any user code, it is just an interface to reach the LLM, it does not have updated information and will inaccurately answer questions such as “What time is it?” (Figure 12). In this section a very simple tool is developed to tell the LLM the current time by invoking a Java code.

Figure 12: The local LLM does not have access to update information

The local tool development steps are described below:

1. Create the class: Create a file DateTimeTool.java in directory src/main/java/org/acme/DateTimeTool.java  and as the file content declare a public class called DateTimeTool;

2. Configure the Tool: Add the annotation @jakarta.enterprise.context.ApplicationScoped to the class so it will be perceived as a managed class by Quarkus;

3. Develop the tool functionality: Create a method on this class and annotate it with @dev.langchain4j.agent.tool.Tool and use the annotation name to describe the tool, the description Provides the current Date and Time Tool is enough for telling the LLM that it should use this tool to tell user time. The method implementation can simply return java.time.LocalDateTime.now().toString(). The full code is shown in Figure 13;

Figure 13: The tool full code.

4. Register the Tool: Open the class AiService that was created in step 5.2.d and register the tool using the annotation tools field and pass the DateTimeTool.class as parameter. The update code of the class AiServices is shown in Figure 14. The tool registration can be checked at the Dev Console accessible on URL http://localhost:8180/q/dev-ui/quarkus-langchain4j-core/tools as illustrated in Figure 15;

Figure 14: The AiService class updated code.

Figure 15: The tool registered for the AiService.

5. Test the tool: Open the chat with the LLM on the URL http://localhost:8180/q/dev-ui/quarkus-langchain4j-core/chat and ask “what time is it?”. It should invoke the tool and return the updated time. Check Figure 16 to see the chat output

Figure 16: Chat with the LLM now with Tool calling

5.4 MCP Setup

Quarkus Langchain4j has MCP support for both client and server, meaning that the application can either consume MCP server made in any technology or serve tools using an external develop MCP Server. For this tutorial section the MCP Java Database Connectivity (JDBC) server is used to handle databases using the well known Java JDBC API. In this case the application will act as client and the existing Tool will be exposed as MCP, making it also a MCP Server. The JDBC MCP server source code is accessible at https://github.com/quarkiverse/quarkus-mcp-servers/blob/main/jdbc/

The steps to setup MCP for the Java application are:

1. Install the required software: A MCP server must be started somehow and the client MCP application must either have the instructions to start the server or call it remotely. The JDBC MCP server is started using jbang that can be downloaded on its website: https://www.jbang.dev/download/

2. Configure the MCP: Open the file src/resources/application.properties and configure the transport type and the command to start the server. It is required to give the MCP a name. In this tutorial the following properties are configured

1. quarkus.langchain4j.mcp,{name}.transport-type: It defines the transport type to communicate with the MCP server. For a local MCP server one must use stdio (Standard I/O) as the value

2. quarkus.langchain4j.mcp.{name}.command: the command to start the MCP server with arguments separated by comma

In this tutorial a MCP server todo-db is created and it uses JDBC to store data on a local file todo.sqlite using SQLite Database. Since MCP are tools definitions injected to the LLM prompts, the timeout is also increased because bigger prompts results in bigger response times. The properties to configure it are as following:

quarkus.langchain4j.openai.timeout=5m

quarkus.langchain4j.mcp.todo-db.transport-type=stdio

quarkus.langchain4j.mcp.todo-db.command=jbang,jdbc@quarkiverse/quarkus-mcp-servers,jdbc:sqlite:todo.sqlite

3. Create the internal MCP Server: The DateTimeTool is already exposed as MCP because the application has the MCP Server extension. It is possible to invoke the tool as MCP using the DEV UI MCP Server tool (http://localhost:8180/q/dev-ui/quarkus-mcp-server-http/tools) as seen in Figure 17. However, the previous invocation of this tool was not using MCP, it was a code call inside the JVM. To call this tool as MCP by the application one must also register it on application.properties:

quarkus.langchain4j.mcp.internal-tools.transport-type=streamable-http

quarkus.langchain4j.mcp.internal-tools.url=http://localhost:8180/mcp/

Figure 17: Invoking the Date Time tool as MCP

4. Verify MCP client registration: Start the Quarkus application (if not running already) and check the DEV UI pane at URL http://localhost:8180/q/dev-ui/quarkus-langchain4j-mcp/mcp-clients. The MCP client todo-db and internal-tools and its tools are listed on this page as displayed in Figure 18.

Figure 18: The JDBC MCP Tools

5. Chat with the LLM: Going to the chat web page (http://localhost:8180/q/dev-ui/quarkus-langchain4j-core/chat) and talking to the LLM will now allow users to have access to the MCP tools. A sample conversation with MCP tools can be found on Figure 19.

Figure 19: Chatting to the LLM now involves access to the MCP tools

5.5. Building the application and improving performance

The Java application is built and assembled into a JAR (Java Archive), which means that users willing to run it will need the Java Virtual Machine (JVM) and that the performance may not be better to other development platforms. The following steps show how to build the application for production use and how to improve performance using the native compilation:

1. Build the application: Building the application results in a JAR file that can be used in the production environment. To build the Java application the command mvnw clean package can be used. the application executable folder target/quarkus-app/ will be ready to run by using the JAR quarkus-run.jar. This JAR file can be executed using the command java -jar target/quarkus-app/quarkus-run.jar as illustrated in Figure 20. The Java application running on JVM can have better performance using JVM flags, but it is out of the scope of this tutorial to tune the Java application. One must make sure that the Quarkus application is not running in development mode to avoid HTTP ports conflict.

Figure 20: Starting the built Java application

2. Check JAR application memory usage: It is possible to check the running Java using jps  to retrieve the process ID, and then reading the process status from the disk  using  the command grep -E "VmRSS|VmHWM" /proc/<PID>/status to retrieve the actual memory used by the process (RSS) as shown in Figure 21. For this application, the non native Java application is running with 168536kb of memory usage (note: the value memory usage may vary according to the system hardware)

Figure 21: Checking the amount of memory used by the application running on the JVM

3. Build native binaries: It is possible to build the Java application as a native binary compatible with cloud environments. The command “./mvnw package -Pnative” downloads the necessary tools and builds the native executable for the Java application. This build process is slower than the one discussed in the previous section, but it removes the need of a JVM for production. A binary file can be found on “target” directory and it is self-contained, meaning that no other dependency is needed. The file can be executed directly and the Quarkus application starts as shown in Figure 22. 

Figure 22: Native application startup

4. Check native application performance:  From the logs in Figure 20 and comparing to Figure 22, it is possible to see that the native compilation takes ~3ms to start while the Java application takes ~100ms. The utility pgrep can be used to retrieve the running native process id and using the same method used for the JAR build we can find the amount of actual memory used by the process which is 51268kb, as shown in Figure 23. 

Figure 23: The running native process memory usage

5.6 Showcase: TODO list application

At the end of this section the Java application will become a TODO list application backed by a database. This will be done by using Tool Calling and MCP. The following steps must be followed to achieve this:

1. Instruct the LLM: Instruct the LLM that it is a TODO application using a system message. The system message can be configured using the annotation dev.langchain4j.service.SystemMessage on the AiService interface; The system message is the following:

        You are the user TODO list manager. For this you have access to the database and should generate SQL commands according to user queries.. Make it sure that the table TODO is created with the columns:

        1. ID: A sequential internal ID for the TODO task

        2. Created: The date when the TODO was created - Use the DateTimeTool to retrieve the currrent date time

        3. Task: A text column for theTODO task description

        4. Completed: a boolean to indicate if the user either completed this task or not

        Ensure the table TODO is created. If a TODO table is already present, no action is needed.

        Make it sure that you follow the primary goal of being a TODO List manager and avoid talking about subjects not related to it. Users may not be technical, hence avoid using technical words such as SQL, Database and so on.

2. Test the application: Start the application dev mode and go to the chat DEV UI Screen (http://localhost:8180/q/dev-ui/quarkus-langchain4j-core/chat). Users should now be able to add TODO items to the list as shown in Figure 24. CHecking the tool call it is possible to see it is invoking the external MCP Server

Figure 24: The chat is now a Todo list

When adding a new item it is possible to see that it invoke both externally developed MCP and internally developed MCP server (Figure 25)

Figure 25: The application uses both external and internal MCP.

6. Conclusion, Limitations, and Future Work

6.1. Conclusion

This tutorial introduced important concepts and guided the user to develop an application that makes use of a local privacy budget LLM connected to enterprise Java code using open protocols and standards.

The source code for this tutorial is available on: https://github.com/jesuino/java-ml-projects/tree/master/java-tool-call-mcp-tutorial

6.2 Limitations

Local LLMs may require hardware that is not available for enterprise scale applications. Its performance may not be comparable to huge LLMs and it requires expertise for selecting the right LLM for the available hardware constraints. A fine-tuned LLM may fit best with enterprise needs than a generic one.

Tool Calling and MCP add performance degradation over time due the high number of tokens added to the context for each user message. LLM hallucinations, although decreased due the use of tools, may lead to incorrect tool calls, hence users must be careful when adding MCP to their applications.

6.3. Future Work

Performance Comparison: Java application performance can be compared with other platforms, such as Go and Python.

Security Evaluation: Evaluate how given MCP and tools access to an application may comprise security; explore jail-breaking possibilities.

Security Hardening: add RBAC, ask the user permission to execute a tool, add railguards for the tool execution

Experiment with LLM: Try the tutorial with different LLMs: fine-tuned, smaller, bigger. Compare local LLM performance with Bigger providers LLMs.

7. References

BRYNJOLFSSON, Erik; LI, Danielle; RAYMOND, Lindsey R. Generative AI at Work. arXiv, 2023. Available at: https://arxiv.org/abs/2304.11771. Accessed on: July 17, 2026.

GONG, Ruihao et al. A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms. arXiv, 2024. Available at: https://arxiv.org/pdf/2409.16694v3. Accessed on: July 17, 2026.

HUGGING FACE. unsloth/Qwen3.6-35B-A3B-GGUF. [n.d.]. Available at: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF. Accessed on: July 17, 2026.

MODEL CONTEXT PROTOCOL (MCP). Getting Started: Introduction. [n.d.]. Available at: https://modelcontextprotocol.io/docs/getting-started/intro. Accessed on: July 17, 2026.

QUARKUS. Building a Native Executable. [n.d.]. Available at: https://quarkus.io/guides/building-native-image. Accessed on: July 17, 2026.

CUMMINS, Holly. New Benchmarks. Quarkus Blog, [n.d.]. Available at: https://quarkus.io/blog/new-benchmarks/. Accessed on: July 17, 2026.

QUARKUS. Quarkus LangChain4j Documentation. [n.d.]. Available at: https://docs.quarkiverse.io/quarkus-langchain4j/dev/index.html. Accessed on: July 17, 2026.

JERROM, Robbie. The rise of small language models in enterprise AI. Red Hat Blog, [n.d.]. Available at: https://www.redhat.com/en/blog/rise-small-language-models-enterprise-ai. Accessed on: July 17, 2026.

ARSLAN, Muhammad et al. Large language models for business and management applications: A review. Information Processing & Management, 2026. Available at: https://www.sciencedirect.com/science/article/pii/S0306457326002554#sec0001. Accessed on: July 17, 2026.

SECOND TALENT. PyTorch vs TensorFlow: Usage, Popularity, and Performance. [n.d.]. Available at: https://www.secondtalent.com/resources/pytorch-vs-tensorflow-usage-popularity-and-performance/. Accessed on: July 17, 2026.

VAADIN. Java for Enterprise Applications. Vaadin Blog, [n.d.]. Available at: https://vaadin.com/blog/java-for-enterprise-applications. Accessed on: July 17, 2026.