Ansible: How to udpate dictionary key value in a file

Question

Alright, thought it was going to be wasy, but I ended up scratching my head.

I have this tracker.txt file, in a dictionary style formatting:

/oracle/db/19.0.0:
  rollback:
    30621255: 
    29213893: 
    29867728: 
    29802382: 
    28318139: 
  apply:
    28318139: 
    29213893: 
    28788272: 
    31431771: 
    32044280: 

I am trying to make the file contents look like this in the end:

/oracle/db/19.0.0:
  rollback:
    30621255: Success
    29213893: Success
    29867728: Success
    29802382: Success
    28318139: Success
  apply:
    28318139: 
    29213893: 
    28788272: 
    31431771: 
    32044280: 

Here is my uf.yml playbook:

---
- hosts: localhost
  gather_facts: no
  vars:
    oracle_home: /oracle/db/19.0.0
    rblist: [30621255, 29213893, 29867728, 29802382, 28318139]
  tasks:
    - name: update file
      include: upd.yml
      with_items:
        - "{{ rblist | default([]) }}"
      loop_control:
        loop_var: plist_item
      when: plist_item | default([])

and this is the upd.yml included task file

---
- debug: msg={{ plist_item }}
- name: update the status
  local_action replace: path=tracker.txt regexp="{{ plist_item }}:" replace="{{ plist_item }}: Success" after='rollback' before='apply'

But I get errors when I run this.

[oracle@anstrlsrv lib]$ ansible-playbook uf.yml

PLAY [localhost] ************************************************************************************************************************************************************************************************************************

TASK [update file] **********************************************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"reason": "We were unable to read either as JSON nor YAML, these are the errors we got from each:
JSON: No JSON object could be decoded

Syntax Error while loading YAML.
  mapping values are not allowed in this context

The error appears to be in '/stage/ap/ansible/lib/upd.yml': line 4, column 94, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

- name: update the status
  local_action replace: path=tracker.txt regexp="{{ plist_item }}:" replace="{{ plist_item }}: Success" after='rollback' before='apply'
                                                                                             ^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:

    with_items:
      - {{ foo }}

Should be written as:

    with_items:
      - "{{ foo }}"
"}
fatal: [localhost]: FAILED! => {"reason": "We were unable to read either as JSON nor YAML, these are the errors we got from each:
JSON: No JSON object could be decoded

Syntax Error while loading YAML.
  mapping values are not allowed in this context

The error appears to be in '/stage/ap/ansible/lib/upd.yml': line 4, column 94, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

- name: update the status
  local_action replace: path=tracker.txt regexp="{{ plist_item }}:" replace="{{ plist_item }}: Success" after='rollback' before='apply'
                                                                                             ^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:

    with_items:
      - {{ foo }}

Should be written as:

    with_items:
      - "{{ foo }}"
"}
fatal: [localhost]: FAILED! => {"reason": "We were unable to read either as JSON nor YAML, these are the errors we got from each:
JSON: No JSON object could be decoded

Syntax Error while loading YAML.
  mapping values are not allowed in this context

The error appears to be in '/stage/ap/ansible/lib/upd.yml': line 4, column 94, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

- name: update the status
  local_action replace: path=tracker.txt regexp="{{ plist_item }}:" replace="{{ plist_item }}: Success" after='rollback' before='apply'
                                                                                             ^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:

    with_items:
      - {{ foo }}

Should be written as:

    with_items:
      - "{{ foo }}"
"}
fatal: [localhost]: FAILED! => {"reason": "We were unable to read either as JSON nor YAML, these are the errors we got from each:
JSON: No JSON object could be decoded

Syntax Error while loading YAML.
  mapping values are not allowed in this context

The error appears to be in '/stage/ap/ansible/lib/upd.yml': line 4, column 94, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

- name: update the status
  local_action replace: path=tracker.txt regexp="{{ plist_item }}:" replace="{{ plist_item }}: Success" after='rollback' before='apply'
                                                                                             ^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:

    with_items:
      - {{ foo }}

Should be written as:

    with_items:
      - "{{ foo }}"
"}
fatal: [localhost]: FAILED! => {"reason": "We were unable to read either as JSON nor YAML, these are the errors we got from each:
JSON: No JSON object could be decoded

Syntax Error while loading YAML.
  mapping values are not allowed in this context

The error appears to be in '/stage/ap/ansible/lib/upd.yml': line 4, column 94, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

- name: update the status
  local_action replace: path=tracker.txt regexp="{{ plist_item }}:" replace="{{ plist_item }}: Success" after='rollback' before='apply'
                                                                                             ^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:

    with_items:
      - {{ foo }}

Should be written as:

    with_items:
      - "{{ foo }}"
"}

PLAY RECAP ******************************************************************************************************************************************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=5    skipped=0    rescued=0    ignored=0   

Not sure where I am going wrong. Another option I thought was something that is documented here, but haven't ventured into it as I dont completely understand it yet: https://www.jeffgeerling.com/blog/2017/changing-deeply-nested-dict-variable-ansible-playbook

Appreciate any help.

Answer 1

The playbook below

- hosts: localhost
  gather_facts: false
  vars:
    oracle_home: /oracle/db/19.0.0
    rblist: [30621255, 29213893]
  tasks:
    - include_vars:
        file: tracker.txt
        name: tracker
    - copy:
        dest: tracker.txt
        content: |
          {
          '{{ oracle_home }}':
          {{ tracker[oracle_home]|combine({'rollback': rollback}) }}
          }

      vars:
        rsuccess: "{{ dict(rblist|product(['success'])) }}"
        rollback: "{{ tracker[oracle_home]['rollback']|combine(rsuccess) }}"

- hosts: localhost
  gather_facts: false
  tasks:
    - include_vars:
        file: tracker.txt
        name: tracker
    - debug:
        var: tracker

gives

  tracker:
    /oracle/db/19.0.0:
      apply:
        '28318139': null
        '28788272': null
        '29213893': null
        '31431771': null
        '32044280': null
      rollback:
        '28318139': null
        '29213893': success
        '29802382': null
        '29867728': null
        '30621255': success

The file is stored in JSON

shell> cat tracker.txt

{"/oracle/db/19.0.0": {"rollback": {"30621255": "success", "29213893": "success", "29867728": null, "29802382": null, "28318139": null, "30621255": "success", "29213893": "success"}, "apply": {"28318139": null, "29213893": null, "28788272": null, "31431771": null, "32044280": null}}}

---

Q: "Retain the file track.txt the way ... since its readability is better."

A: It is possible. There are 2 options on how to use Jinja. Create the complete dictionary tracker and use filter to_nice_yaml

    - copy:
        dest: tracker.txt
        content: |
          {{ tracker|to_nice_yaml }}

The next option is to format the content by indent and to_nice_yaml. Both options would make the code more complex and error-prone. I'd keep JSON. There is a plethora of options on how to display JSON. For example

shell> cat tracker.txt | jq .
{
  "/oracle/db/19.0.0": {
    "rollback": {
      "30621255": "success",
      "29213893": "success",
      "29867728": null,
      "29802382": null,
      "28318139": null
    },
    "apply": {
      "28318139": null,
      "29213893": null,
      "28788272": null,
      "31431771": null,
      "32044280": null
    }
  }
}

---

"Understand the code"

1. Read the file and put the variables into the dictionary tracker

    - include_vars:
        file: tracker.txt
        name: tracker
    - debug:
        var: tracker

gives

  tracker:
    /oracle/db/19.0.0:
      apply:
        28318139: null
        28788272: null
        29213893: null
        31431771: null
        32044280: null
      rollback:
        28318139: null
        29213893: null
        29802382: null
        29867728: null
        30621255: null

2. Create the dictionary of succeeded patches

    - debug:
        var: rsuccess
      vars:
        rsuccess: "{{ dict(rblist|product(['success'])) }}"

gives

  rsuccess:
    29213893: success
    30621255: success

3. Combine the rollback dictionary with rsuccess

    - debug:
        msg: "{{ rollback }}"
      vars:
        rsuccess: "{{ dict(rblist|product(['success'])) }}"
        rollback: "{{ tracker[oracle_home]['rollback']|combine(rsuccess) }}"

gives

  rollback:
    28318139: null
    29213893: success
    29802382: null
    29867728: null
    30621255: success

4. Combine updated rollback with oracle_home

    - debug:
        msg: "{{ tracker[oracle_home]|combine({'rollback': rollback}) }}"
      vars:
        rsuccess: "{{ dict(rblist|product(['success'])) }}"
        rollback: "{{ tracker[oracle_home]['rollback']|combine(rsuccess) }}"

gives

  msg:
    apply:
      28318139: null
      28788272: null
      29213893: null
      31431771: null
      32044280: null
    rollback:
      28318139: null
      29213893: success
      29802382: null
      29867728: null
      30621255: success

5. Create the text block

    - debug:
        msg: |
          {
          '{{ oracle_home }}':
          {{ tracker[oracle_home]|combine({'rollback': rollback}) }}
          }

      vars:
        rsuccess: "{{ dict(rblist|product(['success'])) }}"
        rollback: "{{ tracker[oracle_home]['rollback']|combine(rsuccess) }}"

gives

  msg:
    /oracle/db/19.0.0:
      apply:
        28318139: null
        28788272: null
        29213893: null
        31431771: null
        32044280: null
      rollback:
        28318139: null
        29213893: success
        29802382: null
        29867728: null
        30621255: success

---

Q: with_items: "{{ rblist }}" "tracker[oracle_home]['rollback'][item].item": "VARIABLE IS NOT DEFINED!"

A: The items of rblist are integers. For some reason, the indexes do not evaluate to strings by default. Probably because of possible expected arithmetic inside the indexes. But, the keys in JSON have been stored as strings. For example, the task below

    - debug:
        msg: "{{ item }} {{ item|type_debug }}"
      loop: "{{ tracker[oracle_home]['rollback'].keys()|list }}"
      vars:
        oracle_home: /oracle/db/19.0.0

gives

  msg: 30621255 str
  msg: 29213893 str
  msg: 29867728 str
  msg: 29802382 str
  msg: 28318139 str

It's necessary to cast the keys to string. For example, the task below

    - debug:
        var: tracker[oracle_home]['rollback'][item|string]
      loop: "{{ rblist }}"
      vars:
        oracle_home: /oracle/db/19.0.0
        rblist: [30621255, 29213893]

gives

  tracker[oracle_home]['rollback'][item|string]: success
  tracker[oracle_home]['rollback'][item|string]: success