How to convert a YAML file to a flattened JSON map in Java

So you need to convert a .yaml file to a flattened JSON map in Java? I won’t ask why.

Requirements/What I am using:

• IntelliJ
• Gradle
• Java 11

This is how to convert a YAML file to a flattened JSON map.

Here is a random example I grabbed from GitHub (https://raw.githubusercontent.com/Praqma/helmsman/master/examples/example.yaml taken from Praqma/helmsman: Helm Charts as Code (github.com))

# version: v3.4.0

# context defines the context of this Desired State File.
# It is used to allow Helmsman identify which releases are managed by which DSF.
# Therefore, it is important that each DSF uses a unique context.
context: test-infra   # defaults to "default" if not provided

# metadata -- add as many key/value pairs as you want
metadata:
  org: "example.com/$ORG_PATH/"
  maintainer: "k8s-admin (me@example.com)"
  description: "example Desired State File for demo purposes."
  key: ${VALUE}

# paths to the certificate for connecting to the cluster
# You can skip this if you use Helmsman on a machine with kubectl already connected to your k8s cluster.
# you have to use exact key names here : 'caCrt' for certificate and 'caKey' for the key and caClient for the client certificate
# certificates:
  #caClient: "gs://mybucket/client.crt" # GCS bucket path
  #caCrt: "s3://mybucket/ca.crt" # S3 bucket path
  #caKey: "../ca.key" # valid local file relative path

settings:
  kubeContext: "minikube" # will try connect to this context first, if it does not exist, it will be created using the details below
  #username: "admin"
  #password: "$K8S_PASSWORD" # the name of an environment variable containing the k8s password
  #clusterURI: "$SET_URI" # the name of an environment variable containing the cluster API
  #clusterURI: "https://192.168.99.100:8443" # equivalent to the above
  #storageBackend: "secret"
  #slackWebhook:  "$slack" # or your slack webhook url
  #reverseDelete: false # reverse the priorities on delete
  #### to use bearer token:
  #  bearerToken: true
  #  clusterURI: "https://kubernetes.default"
  # globalHooks:
  #   successCondition: "Initialized"
  #   deleteOnSuccess: true
  #   postInstall: "job.yaml"
  globalMaxHistory: 5

# define your environments and their k8s namespaces
namespaces:
  production:
    protected: true
    limits:
      - type: Container
        default:
          cpu: "300m"
          memory: "200Mi"
        defaultRequest:
          cpu: "200m"
          memory: "100Mi"
      - type: Pod
        max:
          memory: "300Mi"
  staging:
    protected: false
    labels:
      env: "staging"
    quotas:
      limits.cpu: "10"
      limits.memory: "20Gi"
      pods: 25
      requests.cpu: "10"
      requests.memory: "30Gi"
      customQuotas:
        - name: "requests.nvidia.com/gpu"  
          value: "2"    



# define any private/public helm charts repos you would like to get charts from
# syntax: repo_name: "repo_url"
# only private repos hosted in s3 buckets are now supported
helmRepos:
  argo: "https://argoproj.github.io/argo-helm"
  jfrog: "https://charts.jfrog.io"
  #myS3repo: "s3://my-S3-private-repo/charts"
  #myGCSrepo: "gs://my-GCS-private-repo/charts"
  #custom:  "https://$user:$pass@mycustomrepo.org"

# define the desired state of your applications helm charts
# each contains the following:

apps:
    argo:
      namespace: "staging" # maps to the namespace as defined in namespaces above
      enabled: true # change to false if you want to delete this app release empty: false:
      chart: "argo/argo" # changing the chart name means delete and recreate this chart
      version: "0.8.5" # chart version
      ### Optional values below
      valuesFile: "" # leaving it empty uses the default chart values
      test: false
      protected: true
      priority: -3
      wait: true
      hooks:
        successCondition: "Complete"
        successTimeout: "90s"
        deleteOnSuccess: true
        preInstall: "job.yaml"
      #   preInstall: "https://github.com/jetstack/cert-manager/releases/download/v0.14.0/cert-manager.crds.yaml"
      #   postInstall: "https://raw.githubusercontent.com/jetstack/cert-manager/release-0.14/deploy/manifests/00-crds.yaml"
      #   postInstall: "job.yaml"
      #   preUpgrade: "job.yaml"
      #   postUpgrade: "job.yaml"
      #   preDelete: "job.yaml"
      #   postDelete: "job.yaml"
      set:
        "images.tag": $$TAG # $$ is escaped and $TAG is passed literally to images.tag (no env variable expansion)

    artifactory:
      namespace: "production" # maps to the namespace as defined in namespaces above
      enabled: true # change to false if you want to delete this app release empty: false:
      chart: "jfrog/artifactory" # changing the chart name means delete and recreate this chart
      version: "8.3.2" # chart version
      ### Optional values below
      valuesFile: ""
      test: false
      priority: -2
      noHooks: false
      timeout: 300
      maxHistory: 4
      # additional helm flags for this release
      helmFlags: 
        - "--devel"

# See https://github.com/Praqma/helmsman/blob/master/docs/desired_state_specification.md#apps for more apps options

See the result here:

context=test-infra
metadata.org=example.com/$ORG_PATH/
metadata.maintainer=k8s-admin (me@example.com)
metadata.description=example Desired State File for demo purposes.
metadata.key=${VALUE}
settings.kubeContext=minikube
settings.globalMaxHistory=5
namespaces.production.protected=true
namespaces.production.limits[0].type=Container
namespaces.production.limits[0].default.cpu=300m
namespaces.production.limits[0].default.memory=200Mi
namespaces.production.limits[0].defaultRequest.cpu=200m
namespaces.production.limits[0].defaultRequest.memory=100Mi
namespaces.production.limits[1].type=Pod
namespaces.production.limits[1].max.memory=300Mi
namespaces.staging.protected=false
namespaces.staging.labels.env=staging
namespaces.staging.quotas.limits.cpu=10
namespaces.staging.quotas.limits.memory=20Gi
namespaces.staging.quotas.pods=25
namespaces.staging.quotas.requests.cpu=10
namespaces.staging.quotas.requests.memory=30Gi
namespaces.staging.quotas.customQuotas[0].name=requests.nvidia.com/gpu
namespaces.staging.quotas.customQuotas[0].value=2
helmRepos.argo=https://argoproj.github.io/argo-helm
helmRepos.jfrog=https://charts.jfrog.io
apps.argo.namespace=staging
apps.argo.enabled=true
apps.argo.chart=argo/argo
apps.argo.version=0.8.5
apps.argo.valuesFile=
apps.argo.test=false
apps.argo.protected=true
apps.argo.priority=-3
apps.argo.wait=true
apps.argo.hooks.successCondition=Complete
apps.argo.hooks.successTimeout=90s
apps.argo.hooks.deleteOnSuccess=true
apps.argo.hooks.preInstall=job.yaml
apps.argo.set.images.tag=$$TAG
apps.artifactory.namespace=production
apps.artifactory.enabled=true
apps.artifactory.chart=jfrog/artifactory
apps.artifactory.version=8.3.2
apps.artifactory.valuesFile=
apps.artifactory.test=false
apps.artifactory.priority=-2
apps.artifactory.noHooks=false
apps.artifactory.timeout=300
apps.artifactory.maxHistory=4
apps.artifactory.helmFlags[0]=--devel

First we’ll need two dependencies (I used Gradle), add them directly to your build.gradle dependencies like this:

dependencies {
    implementation group: 'org.yaml', name: 'snakeyaml', version: '1.25'
    compile "org.springframework.vault:spring-vault-core:2.1.1.RELEASE"


}

You can get them directly here:

The Central Repository Search Engine (maven.org) (Snake Yaml)

JsonMapFlattener (Spring Vault 2.2.3.RELEASE API)

Ensure they’re imported into IntelliJ or whatever IDE you’re using

Your external libraries should look like this or similar to it

First, we want to create a simple helper class for using SnakeYAML.

First create a java class called SnakeYAMLHelper:

import org.springframework.vault.support.JsonMapFlattener;

import java.io.*;
import java.net.URL;
import java.util.Map;

public class YAMLReader {
    private final Map<String, String> flattenedMap;
    private final String tempname;
    private final StringBuilder content = new StringBuilder();
    private final SnakeYAMLHelper syh = new SnakeYAMLHelper();
    
    public YAMLReader(String url, String tempname) {
        this.tempname = tempname;
        flattenedMap = getFlattenedStringMapURL(url);
    }

    private Map<String, String> getFlattenedStringMapURL(String url){
        return JsonMapFlattener.flattenToStringMap(syh.YamlLoad(loadYamlFromURLtoString(url)));
    }

    public YAMLReader(File file, String tempname) {
        this.tempname = tempname;
        flattenedMap = getFlattenedStringMapFile(file);
    }

    private Map<String, String> getFlattenedStringMapFile(File file){
        return JsonMapFlattener.flattenToStringMap(syh.YamlLoad(loadLocalYaml(file)));
    }
    
    public Map<String, String> getMap(){
        return flattenedMap;
    }

    public void printMap(){
        for(var item : flattenedMap.entrySet()){
            System.out.println(item);
        }
    }

    public void createMapFile(String outputFilePath){
        StringBuilder stringToWrite = new StringBuilder();
        for(var item : flattenedMap.entrySet()){
            stringToWrite.append(item).append("\n");
        }
        File file = new File(outputFilePath);

        try(FileOutputStream fos = new FileOutputStream(file);
            BufferedOutputStream bos = new BufferedOutputStream(fos)) {
            byte[] bytes = stringToWrite.toString().getBytes();
            bos.write(bytes);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    private String loadLocalYaml(File file){
        try (FileInputStream input = new FileInputStream(file)) {
            BufferedReader br = new BufferedReader(new InputStreamReader(input));
            String line;
            while ((line = br.readLine()) != null) {
                content.append(line).append(System.lineSeparator());
            }
        }
        catch (IOException e) {
            e.printStackTrace();
        }
        return content.toString();
    }

    private String loadYamlFromURLtoString(String url){
        try (BufferedInputStream in = new BufferedInputStream(new URL(url).openStream());
             FileOutputStream fileOutputStream = new FileOutputStream(tempname)) {
            byte[] dataBuffer = new byte[1024];
            int bytesRead;
            while ((bytesRead = in.read(dataBuffer, 0, 1024)) != -1) {
                fileOutputStream.write(dataBuffer, 0, bytesRead);
            }
            return loadLocalYaml(new File(tempname));
        } catch (IOException e) {
            e.printStackTrace();
            return "";
        }
    }
}

For this class, there is two ways to use it, one is for converting a local file, one is for converting a URL link such as the one I linked at the top of the article and converting it.

In this article we will show the example of a non local file hosted elsewhere, but feel free to use the local methods for converting a local file.

public class Main {

    public static void main(String[] args){
        final String url = "https://raw.githubusercontent.com/Praqma/helmsman/master/examples/example.yaml";
        String filename = "./test";
        // we use temp file to download and store the downloaded non local .yaml file
        String tempname = "./test_raw";

        YAMLReader yamlR = new YAMLReader(url, tempname);
        yamlR.printMap();
        yamlR.createMapFile(filename);
    }
}

We use create map file to create a local converted file, and we use print map to show case the map in console output.

The output to the console should be the same as above, and there should be a locally created file! There is a get method to operate on the map directly if you need. This code can definitely be improved and if I have time I will maintain this repo here: firefelix/JavaYAMLtoFlatMap: Simple Java code for converting a .yaml file to a flat map in Java (github.com)