Langchain4j re -looking 6 months later

Last year I started digging a little around Langchain4j. This is a fast -growing project and I wanted to get acquainted with the updates. I also wanted to check how to integrate the context protocol server in Langchain4J.

1. Version of beta

I wrote my last post in November 2024 and used the latest version available at that time, v0.35. Langchain4j started his journey to 1.0 last December.

Langchain4j follows Semver. The caregivers used this opportunity to introduce breakage changes. In my case, I had to update my code to take into account the violation of API changes.

Integration of a project reactor

Langchain4J offers the integration of the project reactor; I missed it in my previous music. Kotlin Coroutinsis simplifies the code manyTo.

I am using AiServicesSo I used to define the Langchain4J interface to apply on Runtime:

interface ChatBot {
    fun talk(@MemoryId sessionId: String, @UserMessage message: String): TokenStream
}

We should add the following addiction:

    dev.langchain4j
    langchain4j-reactor
    1.0.0-beta3

Now we can change the return type Flux up to a TokenStreamTo. Here is an updated signature:

interface ChatBot {
    fun talk(@MemoryId sessionId: String, @UserMessage message: String): Flux
}

This makes creation sink Above unnecessary. We can simplify the code as follows:

val flux = chatBot.talk(m.sessionId, m.text)
ServerResponse.ok().bodyAndAwait(flux.asFlow())

Remember that two days of smoothing can easily save two hours of documentation reading! I didn't do the last one.

Integration of a model context protocol server

Until that moment, our changes were minimal. In this section I want to integrate Mcp In my Langchain4J app.

A generation related to the search

Needs a lot and a lot of resources to train Llm: This means direct time and money. For this reason, companies restrict the training of new model versions. The relevance of the model is reduced over time in collecting information and changes, while the LLM database is unchanged. In addition, LLM is trained for public data – nature, while most companies also want to query their private data.

Learned an increased generation was a traditional way to deal with these boundaries. A respected generation of search is a two -step process. In the first phase, the tool parses data, vector, according to LLM and stores it in the vector database; In the second, the tool uses the database as additional data in the LLM query.

The context protocol of the model

The last way to deal with the static nature of the LLMs is MCP.

The MCP is an open protocol that standardizes how applications provide context for LLMs. Think of MCP like the USB-C port of AI applications. Just as the USB-C offers a standardized way to connect its devices to various external devices and accessories, the MCP offers a standardized way to connect AI models to different data sources and tools.

– Start with a model context protocol

MCP has two advantages with a rag:

• The processed data of the RAG are customized for the model. If you want to use a new model, the parsing phase must be re -executed. The MCP standardizes interaction between the customer and the server, making them independent of technology.

• RAG allows you to read data. MCP allows any API -call dynamically to access data or to carry out actions!!

MCP defines two transport alternatives for customer-server communication:

• STIO: Customer launches a subprocess and communication takes place as standard and standard
• Http server-sentence events

The architect of the solution

After the theory above, we are now ready for a practical part. It starts with the selection of the MCP server. Here is a good starting point. However, I chose the official Github MCP server as Langchain4J documentation mentions it.

Githubi MCP server offers stio Transport. This means that we should get a binary and start with its application. This is fast compared to HTTP transport, but given the overall time, which includes the http call of the model and the calculation time on its side, it is not insignificant. From the point of view of architecture, I would prefer a special component with this process.

After some research, I found the MCP-Proxy project. This allows you to switch from either Stio to HTTP or HTTP to STIO. It is also available as a docker image. We can connect both the server and the proxy to the following Dockerfile:

FROM ghcr.io/sparfenyuk/mcp-proxy:latest

ENV VERSION=0.2.0
ENV ARCHIVE_NAME=github-mcp-server_Linux_x86_64.tar.gz

RUN wget  -O /tmp/$ARCHIVE_NAME \ #1
    && tar -xzvf /tmp/$ARCHIVE_NAME -C /opt \                      #2
    && rm /tmp/$ARCHIVE_NAME                                       #3

RUN chmod +x /opt/github-mcp-server                                #4

1. Download the archive
2. Extract it
3. Remove the archive
4. Change to binary to start

Note that we cannot define it CMD Because binary only allows you to configure the port and host with parameters. For this reason, we must post the command during the working hours or in my case docker-compose.yaml:

services:
  mcp-server:
    build:
      context: github-mcp-server
    env_file:
      - .env                                                       #1
    command:
      - --pass-environment                                         #2
      - --sse-port=8080                                            #3
      - --sse-host=0.0.0.0                                         #4
      - --                                                         #5
      - /opt/github-mcp-server                                     #6
      - --toolsets
      - all
      - stdio

1. We need a GITHUB_PERSONAL_ACCESS_TOKEN An environmental variable with a valid sign in github to authentication
2. Transmit all environmental variables to a subprocess
3. Set the list of listening
4. Bind to any IP
5. Buffer Server “connects” to Studio MCP server after a dash
6. Start the server with all the options allowed

The picture gives /sse End -point on Port 8080.

Coding

The coding part is the simplest. Head to Langchain4J documentation in MCP and observe. In the project, this means the following:

bean {
    val transport = HttpMcpTransport.Builder()
        .sseUrl(ref().mcp.url)              //1
        .logRequests(true)                                         //2
        .logResponses(true)                                        //2
        .build()
    val mcpClient = DefaultMcpClient.Builder()
        .transport(transport)
        .build()
    mcpClient.listTools().forEach { println(it) }                  //3
    McpToolProvider.builder()
        .mcpClients(listOf(mcpClient))
        .build()
}
bean {
    coRouter {
        val chatBot = AiServices
            .builder(ChatBot::class.java)
            .streamingChatLanguageModel(ref())
            .chatMemoryProvider { MessageWindowChatMemory.withMaxMessages(40) }
            .contentRetriever(EmbeddingStoreContentRetriever.from(ref>()))
            .toolProvider(ref())                  //4
            .build()
        POST("/")(PromptHandler(chatBot)::handle)
    }
}

1. I added a ConfigurationProperty Class SSE URL to parameter
2. MCP Protocol gives a way to send the logs back to the customer
3. Not necessary but it helped me to make sure the customer is connected to the server and could list the tools offered
4. Connect the above MCP tool provider AiServices

At this point, the model should submit a query that corresponds to some of the registered tools with the MCP server.

curl -N -H 'Content-Type: application/json' localhost:8080 -d '{ "sessionId": "1", "text": "What are my top three most popular GitHub repos?" }'

I tried several times and got the answers according to their lines:

Unfortunately, the provided text does not contain any information about your top three most popular GitHub repositories. The text appears to be a blog post or a personal website, and it mentions some of your projects and experiences with GitHub, but it does not provide any metrics or statistics on the popularity of your repositories.

If you want to know more about the popularity of your GitHub repositories, I would recommend checking out GitHub's own analytics tools, such as GitHub Insights or the Repository Insights API. These tools can provide information about the number of followers, stars, and forks for each repository, as well as other metrics like engagement and activity.

The model simply ignored the tools despite the opposite documentation.

Approval of the solution

I read Langchain4J documentation a couple of times, but to no avail. I tried to use Openai and a handful of other AI tools that had no success. Most of the answers confirmed that it should work out of the box. Some mentioned the tool directly to the tool, which loses its purpose; One mentioned that he was not supported by the tools. I checked to be a blog: it announced the support of tools in 2024. I was stuck for almost a day, wondering what I did wrong.

Dissolved architecture introduces more moving pieces. I suspected that there might be something wrong in the whole chain. I removed the mcp proxy, I added github-mcp-server directly to the image of the application and made the code from HTTP to STIO. It did not solve the problem.

I was going to give up when I decided to come back to the roots. I copied the sample documentation: it just worked! That was my ha-moment.

The test uses to be Openai during the time. I tried MCP Openai, Mistral Ai and to be. Only the Openai model works with MCP. I sent the same query as above:

curl -N -H 'Content-Type: application/json' localhost:8080 -d '{ "sessionId": "1", "text": "What are my top three most popular GitHub repos?" }'

Now Openai correctly maps the correct tool and return the answer I expected:

Here are my findings regarding your top three most popular GitHub repositories:

1. **[opentelemetry-tracing](  
   - **Description**: Demo for end-to-end tracing via OpenTelemetry.  
   - **Stars**: 68  
   - **Forks**: 25  
   - **Open Issues**: 10  

2. **[kaadin](  
   - **Description**: Kotlin DSL for Vaadin.  
   - **Stars**: 44  
   - **Forks**: 12  
   - **Open Issues**: 3  

3. **[jvm-controller](  
   - **Description**: Example on how to write a Kubernetes controller in Java.  
   - **Stars**: 33  
   - **Forks**: 10  
   - **Open Issues**: 0  

These repositories demonstrate a range of your interests and contributions in the areas of observability, Kotlin development, and Kubernetes.%    

As we forward the authentication mark to the MCP server, which transmits it to the Github API, the latter knows which user is calling. So it can be interpreted my repos Part of the above request. I admit that this is an unusual use of uses for conventional web applications that correspond to several users, but each use one authentication mark. However, it is ideal for a desktop application.

Other regular questions, e.g.Find the most popular repositories in Github, are appropriate for web applications because they have no indirect context – user.

Conclusion

The main focus of this post is the integration of the MCP server into the Langchain4J application. Although the configuration is straightforward due to documentation, there are some warnings.

First of all, the way the MCP server fits in your architecture is still up to you. I had to be creative to divorce using great mcp-proxyTo. The Langchain4J then seems to be a leaking abstraction. This allows you to give you a strong layer of abstraction, but the applications underneath it protect you equal. I want the documentation to mention this, even if I realize that the current version is in the beta.

All in all it was a fun ride. I learned from MCP in the real world and it opened quite a few doors for project ideas.

The full source code of this post can be found in Github.

To go further:

Originally published Java Geek on April 27, 2025