kotlin - how to compile kotlinx.serialization libraries on the command line?

Question

This is very close to what I'm trying to accomplish. How to compile and run kotlin program in command line with external java library

I really want to learn how to compile and run simple code that includes libraries but am getting a bit lost when it comes to including classpaths.

I’m currently trying to compile and run

import kotlinx.serialization.*
import kotlinx.serialization.json.*


@Serializable 
data class Project(val name: String, val language: String)

fun main() {
    // Serializing objects
    val data = Project("kotlinx.serialization", "Kotlin")
    val string = Json.encodeToString(data)  
    println(string) // {"name":"kotlinx.serialization","language":"Kotlin"} 
    // Deserializing back into objects
    val obj = Json.decodeFromString<Project>(string)
    println(obj) // Project(name=kotlinx.serialization, language=Kotlin)
}

using

kotlinc -cp "C:PROGRA~1Kotlinlibkotlinx-serialization-runtime-1.0-M1-1.4.0-rc.jar" main.kt

to compile with this compiler

https://blog.jetbrains.com/kotlin/2020/07/kotlin-1-4-rc-released/

allowed lib at bottom of the article that's where kotlinx-serialization-runtime-1.0-M1-1.4.0-rc.jar is coming from. I chose this runtime jar because when I use the new kotlin 4.0.21 compiler it requires the kotlin-serialization-runtime-1.0.1.jar which you need to build yourself but when I download the source and run gradle build it doesn't seem to get generated (separate problem but would love to know how to build the runtime jar myself)

when I try and run I get

Exception in thread "main" java.lang.NoClassDefFoundError: kotlinx/serialization/json/Json
        at MainKt.main(main.kt:12)
        at MainKt.main(main.kt)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at org.jetbrains.kotlin.runner.AbstractRunner.run(runners.kt:64)
        at org.jetbrains.kotlin.runner.Main.run(Main.kt:149)
        at org.jetbrains.kotlin.runner.Main.main(Main.kt:159)
Caused by: java.lang.ClassNotFoundException: kotlinx.serialization.json.Json
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:435)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        ... 9 more

I know I need to include a classpath when I run

kotlin MainKt 

but have tried everything with no success

I've tried many different combinations of things including compiling with and without -Xplugin="C:PROGRA~1Kotlinlibkotlinx-serialization-compiler-plugin.jar doesn't seem to make a difference so I left it off.

I have tried compiling to both a java .jar as well as a kotlin .class file both seem to need classpath information at runtime. I would rather compile to a kotlin .class and keep java out of this until I really need it. This way I can learn what java is really doing in my application.

I guess what I really want to know is how one can determine what is required at runtime for an executable to run. I found this site which helps show dependencies but is for older versions of kotlin https://kotlin.binarydoc.org/org.jetbrains.kotlin/kotlin-compiler-dist/1.3.71/package?package=kotlinx.serialization

I’ve also been peaking into the .class files using https://github.com/google/android-classyshark and https://github.com/borisf/classyshark-bytecode-viewer

but still when people tell others, on StackOverflow, what classpath they need to use to solve their problem it seems like magic. Can someone out there teach me how to fish without gradle?

p.s. If anyone has any good resources on learning the internals of how gradle is building the project. I've looked here a bit https://docs.gradle.org/current/userguide/userguide.pdf but didn’t seem to help. maybe I missed something. Also, this page https://kotlinlang.org/docs/reference/serialization.html#example-json-serialization seems to have what I need but can't seem to transfer that to what the command line needs.

dependencies {
     implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.0.1")
 }

what does this mean? I think this is referring to this https://github.com/Kotlin/kotlinx.serialization

but then how can I build what I need from this repo and use it to allow my application to know where its runtime dependencies are? ugh. I really wanted to figure all this out myself, but I must kneel to the internet gods for this one. Sorry, my post is a mess. I love to learn.

question from:https://stackoverflow.com/questions/65649334/how-to-compile-kotlinx-serialization-libraries-on-the-command-line

Answer 1

It took some time but I was able to build and run the serialization sample found at https://github.com/Kotlin/kotlinx.serialization on the command line using the current kotlinc compiler and the kotlinx.serializtion.1.0.1 library.

Here are the direct links to the compilers and libs

kotlinc and kotlinc-native v1.4.20 https://github.com/JetBrains/kotlin/releases/tag/v1.4.20

Kotlinx.serialization v1.0.1 https://github.com/Kotlin/kotlinx.serialization/releases/tag/v1.0.1

These both can also be found in the 1.4.20 releases blog post under the section titled How To Update: https://blog.jetbrains.com/kotlin/2020/11/kotlin-1-4-20-released/

Setting Up Katlin’s .jar Libraries

---

After updating my path to point to the new compilers I still needed to build the serialization libs. This was as simple as running gradle build in the root directory of the unzipped kotlinx-serialization-1.0.1 folder. Make sure to set your JAVA-HOME system variable before you do this or it won’t work.

Once it's built you need to grab both the kotlinx-serialization-json-jvm-SNAPSHOT-1.0.1.jar and the kotlinx-serialization-core-jvm-SNAPSHOT-1.0.1.jar files and move them into the project directory. This definitely confused me because I had found a runtime lib for kotlinx serialization on the MVN repository site that was one jar file, but I wasn't seeing it after building the 1.0.1 libraries. Once I extracted the 1.0.1 runtime jar I found online, by renaming the .jar to .zip, it became apparent that it consisted of both the contents of the core and json jars. Don’t use the kotlinx-serialization-1.0.1-SNAPSHOT.jar. This jar only contains a blank MANIFEST.ms file. You can find the kotlinx-serialization-core-jvm-1.0.1-SNAPSHOT.jar in the kotlinx.serialization-1.0.1coreuildlibs folder and the kotlinx-serialization-json-jvm-1.0.1-SNAPSHOT.jar in the kotlinx.serialization-1.0.1formatsjsonuildlibs folder. anyways.

Compiling Your .jar Library

---

once you have the jars in your project folder you can build your project I included my cleanbuildandrun.sh shell script down below for easy reference. My first attempt 1) was to try and build the project without compiling it to a .jar library file. This was a complete failure. I got it to compile but running the project proved much harder. I was unable to tell kotlin where the libraries were at runtime. I tried so many different things Including trying to point it to a manifest file I created but nothing seemed to work. It seems you need to build an executable jar in order to make this work. which brings me to my second try 2). This is where I found more success.

Attempt 2)

• First you need to include the kotlinx-serialization-compiler-plugin.jar using the "-Xplugin" compiler flag. My understanding is that plugins are used to define annotations to the compiler like @Serializable. You can find this jar file in the lib folder inside the compiler you just downloaded. I copied this into my projects /lib folder next to the other jar files to make things self-contained and portable.

• Next you need to tell the compiler where to find the library classes you want to access using the "-classpath" or "-cp" compiler flag.

• Make sure to include kotlin runtime libraries using the "-include-runtime" compiler flag. This will bundle the kotlin standard class libraries within your jar so you don’t need to point at them during runtime.

• Last direct the compiler to build a jar file by providing the -d compiler flag with the name and extension of your soon to be .jar file. That’s it, your off compiling.

Example Shell Script:

#!/bin/bash

sh clean.sh
case $1 in
    1) # Comming Soon
        kotlinc -verbose -Xplugin="libkotlinx-serialization-compiler-plugin.jar" 
                         -cp "libkotlinx-serialization-json-jvm-1.0.1.jar;libkotlinx-serialization-core-jvm-1.0.1.jar" 
                          main.kt
        ;;
    2) # Working
        kotlinc main.kt -Xplugin="libkotlinx-serialization-compiler-plugin.jar" 
                        -cp "libkotlinx-serialization-json-jvm-1.0.1.jar;libkotlinx-serialization-core-jvm-1.0.1.jar" 
                        -include-runtime -d main.jar
                         jar ufm Main.jar ManifestAdditions.txt lib
                         kotlin main.jar
        ;;
    3) # Comming Soon
        kotlinc-native main.kt -verbose -Xplugin="libkotlinx-serialization-compiler-plugin.jar" 
                        -cp "libkotlinx-serialization-json-jvm-1.0.1.jar;libkotlinx-serialization-core-jvm-1.0.1.jar" 
                        -manifest ManifestAddition.txt -o main
        ;;
esac

Running your .jar Library

---

By default, when you compiled the jar it created a MANIFEST.ms file that it uses to tell your jar library where the entry point is. which would be enough if we weren’t using additional libraries in our application. So next we need to add those libraries to the jar file we compiled while at the same time updating its MANIFEST.ms file to tell it where those libraries are within that jar file. We can use the cli tool jar to accomplish this. With the command:

jar ufm Main.jar ManifestAdditions.txt lib

we are able to update the current jar file.

• u - tells jar we want to update an existing jar file
• f - indicates that we are providing the jar file we want to update on the cmd line
• m - indicates that we will be providing the manifest file

The Manifest .txt file should look like this:

Main-Class: MainKt
Class-Path: libkotlinx-serialization-core-jvm-1.0.1.jar libkotlinx-serialization-json-jvm-1.0.1.jar

Make sure to add a new line at the end of the file or it won’t parse the Class-Path.

That’s it. Now we have an executable jar file that we can use to run our serialization code on the command line:

kotlin main.jar

should output:

{"name":"kotlinx.serialization","language":"Kotlin"}
Project(name=kotlinx.serialization, language=Kotlin)

Post Mark

---

I would really like to turn this answer into a blog post that explains how to use the kotlin compiler, with libraries, on the command line. The information is out there but it seems to be scattered. I would like to include how to compile and run without using jar files, if that’s even possible, as well as how to compile and run using the native compiler. If anyone can help fill in these gaps it would be much appreciated. I think this information could help others learn how to set up simple test environments so they can better understand the functionality of these libs without having to set up a build script. This will be my first attempt at creating a tutorial type blog post so any information would really help.