Playing the JSONata the iXML Way

[rdiger_plantiko2]

• 
  Producing the Result Piece by Piece
  


• 
  Ideas for Improvements
  


• 
  Inserting XML Elements from Other Documents
  


• 
  Using Method Chaining
  


• 
  Adding Arbitrary Data As String
  


• 
  Crash Early - Using Assertions for the Document Structure
  


• 
  Getting the Result
  


• 
  Inspecting the iXML Document in the Debugger
  


• 
  Conclusion
  

Usually, a JSON string will be created at once in a final step of request processing. But in some cases, the whole data object has to be composed from pieces. In this case, a JSON writer instead of a transformation will be more appropriate. In this blog, I show a writer object that works internally with an iXML DOM and has a more flexible interface than the sXML string writer recommended in the docu for this purpose.

Producing the Result Piece by Piece

The ABAP documentation shows how to produce JSON piece by piece with a string writer - to be precise: with class cl_sxml_string_writer. In the end, it is always this class that is needed to produce a result. But this doesn't mean that they are required for the process of building the JSON-XML document as well. Any way to produce an XML document is viable.

In the demo program of the ABAP documentation, an auxiliary method write_element() is used to propagate data into the JSON-XML in being:

method write_element.

  writer->open_element( name = name ).

  if attr is not initial.

    writer->write_attribute( name = 'name' value = attr ).

  endif.

  if value is not initial.

    writer->write_value( value = value ).

  endif.

endmethod.

With this additional method and the String Writer's proper methods open_element() and close_element(), the ABAP docu gives the following code sample to produce a JSON data object:

write_element( name  = 'object' ).

write_element( name  = 'str'    attr = 'order'

               value = '4711' ).

writer->close_element( ).

write_element( name  = 'object' attr = 'head' ).

write_element( name  = 'str'    attr = 'status'

               value = 'confirmed' ).

writer->close_element( ).

write_element( name  = 'str'    attr = 'date'

               value = '07-19-2012' ).

writer->close_element( ).

writer->close_element( ).

write_element( name  = 'object' attr = 'body' ).

write_element( name  = 'object' attr = 'item' ).

write_element( name  = 'str'    attr = 'units'

               value = '2' ).

writer->close_element( ).

write_element( name  = 'str'    attr = 'price'

               value = '17.00' ).

writer->close_element( ).

write_element( name  = 'str'    attr = 'Part No.'

               value = '0110' ).

writer->close_element( ).

writer->close_element( ).

write_element( name  = 'object' attr = 'item' ).

write_element( name  = 'str'    attr = 'units'

               value = '1' ).

writer->close_element( ).

write_element( name  = 'str'    attr = 'price'

               value = '10.50' ).

writer->close_element( ).

write_element( name  = 'str'    attr = 'Part No.'

               value = '1609' ).

writer->close_element( ).

writer->close_element( ).

write_element( name  = 'object' attr = 'item' ).

write_element( name  = 'str'    attr = 'units'

               value = '5' ).

writer->close_element( ).

write_element( name  = 'str'    attr = 'price'

               value = '12.30' ).

writer->close_element( ).

write_element( name  = 'str'    attr = 'Part No.'

               value = '1710' ).

writer->close_element( ).

writer->close_element( ).

writer->close_element( ).

writer->close_element( ).

Ideas for Improvements

Being aware that this is code from a documentation (which has other standards than productive code), I see some possible improvements:

• The code is too verbose.
• Code concerned with the production of the JSON-XML document, but with no knowledge about the special data structure of the application - like the write_element( ) method above - could be factored out into a separate writer class. 
  
• Nesting errors (i.e. forgetting a writer->close_element() ) should be detected early.
• Parameter naming surprise: Import parameter called `attr` but will be the attribute `name` of the target document.
• Parameter `name` could be eliminated by providing properly named methods in an own object for the seven possible element names.
• It should not be necessary to close simple elements like 'str' explicitly, since by definition they have no content.
• The three members of the body object all have the name item. It's true: non-unique object keys are allowed in the JSON data format, but with an explicit "should not use" warning:
  
  

  The names within an object SHOULD be unique.

Source: The JSON RFC, section 2.2

Indeed, a JSON deserializer in JavaScript must inevitably lose information, if it preserves the source data structure. Usually, only the first scanned member with that name will be transferred to the target. So in this case it would be better to design an items array.

• It's not possible to add XML elements from other documents - for example splice an XSLT transformation result into a larger object.

An improved version would work with a separate general-purpose writer object for producing the JSON-XML.

• The new writer object will have internal state:
  • the XML document in construction,
  • and a pointer to the current element (new children will be appended to the current element).
• This writer carries dedicated methods for
  • opening complex elements (object, array)
  • or adding simple elements (strings, numbers, booleans).
• The simple elements will be closed implicitly. Only for the complex elements, a close...() call is necessary
• open_object() and open_array() have only one parameter, an optional one: The name - if the element is itself a member of an enclosing object. This way, it becomes clear to which open...() a particular close...() refers to.
  • The close...() methods will contain an assert statement to ensure that the document will be properly nested.
• In the example, we switch to using an array of items instead of an object of identically named members.

With a delegated writer object, the above code looks like this:

writer->open_object( )

  writer->add_string( name  = `order`

                      value = `4711` ).

  writer->open_object( `head` ).

    writer->add_string( name  = `status`

                        value = `confirmed` ).

    writer->add_string( name  = `date`

                        value = `07-19-2012` ).

  writer->close_object( `head` ).

  writer->open_object( `body` ).

    writer->open_array( `items` ).

      writer->open_object( ).

        writer->add_string( name = `units`

                            value = `2` ).

        writer->add_string( name = `price`

                            value = `17.00` ).

        writer->add_string( name = `Part No.`

                            value = `0110` ).

      writer->close_object( ).

      writer->open_object( ).

        writer->add_string( name = `units`

                            value = `1` ).

        writer->add_string( name = `price`

                            value = `10.50` ).

        writer->add_string( name = `Part No.`

                            value = `1609` ).

      writer->close_object( ).

      writer->open_object( ).

        writer->add_string( name = `units`

                            value = `5` ).

        writer->add_string( name = `price`

                            value = `12.30` ).

        writer->add_string( name = `Part No.`

                            value = `1710` ).

      writer->close_object( ).

    writer->close_array( `items`).

  writer->close_object( `body`).

writer->close_object( ).                                             

There is no considerable improvement regarding code lines, but, as I think, the result is better readable.

Inserting XML Elements from Other Documents

If the data of the header and of the items table are more complex, it might be an advantage to build them with a transformation. If the writer's target is an iXML instance, not a writer, it will be possible to add iXML elements from those transformations.

The docu example could be written by merging together the results of two transformations - zorder_head and zorder_items:

writer->open_object().

writer->open_object(`head`).

data lo_head type ref to if_ixml_document.

lo_head = cl_ixml=>create( )->create_document( ).

call transformation zorder_head

  source xml order->get_header( )

  result xml lo_head.

writer->add_element( lo_head->get_root_element( ) )

writer->close_object(`head`).

writer->open_object(`body`).

data lo_items type ref to if_ixml_document.

lo_items = cl_ixml=>create( )->create_document( ).

call transformation zorder_items

  source xml order->get_items( )

  result xml lo_items.

writer->add_element( lo_items->get_root_element( )

writer->close_object(`body`).

writer->close_object( ).

In a more real-life example, the code for adding the header and the items would be contained in two separated methods, according to the single responsibility principle (SRP).

Of course, the XML element can stem from any XML document whatever - not necessarily from a transformation's result.  Here is another example - in the form of a unit test:

  method insert_from_xml_source.

    data: lo_element type ref to if_ixml_element.

 

* Extract an element from a test document

    lo_element ?= get_root(

                     `<test><object><str name="test">test</str></object></test>`

                     )->get_first_child( ).

* Use the element as building block for the target

    go_output->open_array( ).

      go_output->add_number( 1 ).

      go_output->add_element( lo_element ).

      go_output->add_number( 2 ).

    go_output->close_array( ).

    assert_result( `[1,{"test":"test"},2]` ).

  endmethod.

Using Method Chaining

To allow for method chaining, all methods have the writer instance itself as return value. This allows to concatenate the calls -  like in this unit test method:

  method simple_array.

    go_output->open_array(

      )->add_boolean( 'X'

      )->add_boolean( ' '

      )->add_string( `test`

      )->add_number( 17

      )->close_array( ).

    assert_result( `[true,false,"test",17]` ).

  endmethod.

In use cases like these, method chaining reduces the code verbosity. It is best readable when extended over several lines, using indentation to display nesting levels, as in the example above.

Putting the line-break inside of the brackets as above, was the only way accepted by my ABAP compiler. I am pretty confident that in future ABAP versions a line break after a closing bracket and/or after the member operator (->) will be possible, which would make the code look more natural.

Adding Arbitrary Data As String

It is useful to have a method which puts an arbitrary ABAP data object into a string. This is little more than applying the built-in MOVE logic - except for numbers. When ABAP-moved to a string, numbers sometimes get an invisible blank at the end: if the datatype admits a sign, this is the place reserved for the minus sign (which follows the number in ABAP). For these cases, trimming away the trailing blanks would be required, as can be read off from the expectation in the following unit test:

  method add_as_string.

    data: lv_n3 type n length 3 value 0.

    go_output->open_array( ).

      go_output->add_as_string( 0 ).

      go_output->add_as_string( 1 ).

      go_output->add_as_string_if_noninitial( 0 ).

      go_output->add_as_string_if_noninitial( 1 ).

      go_output->add_as_string( lv_n3 ).

    go_output->close_array(  ).

    assert_result( `["0","1","1","000"]` ).

  endmethod.

Crash Early - Using Assertions for the Document Structure

"Crash early" is one of the pragmatic programmers' famous tips. Letting the program crash as near as possible at the code location causing the damage, simplifies the analysis of the problem. For our JSON writer, we use two locations to insert an assert statement:

1. When an element is added to the structure.
   The element to which it is added must be either the root or a complex element (<object>,<array>,<member>).
2. When a complex element is closed.
   The element name of the closed element must match the currently open element: It would be wrong to close an object, if the last opened element was an array. Also, the "name" attribute of the closed element, if given, must match the "name" attribute of the currently open object.
   
   

Therefore, the method add_element() is implemented as follows:

method add_element.

  assert id zdev condition add_element_allowed( io_element->get_attribute( `name`) ) eq abap_true.

* Adds an XML element from elsewhere into the tree

  go_current_node->append_child( io_element ).

  eo_me = me.

endmethod.

For the assertion, I use a special checkpoint group ZDEV, which in all our development systems is configured in transaction SAAB to dump when an assertion is violated. The condition itself is implemented in a separated method:

method add_element_allowed.

  data: lv_parent_type type string.

  lv_parent_type = go_current_node->get_name( ).

  ev_allowed = boolc(

* Adding a new element is allowed only to

* 1.) the root node, or

    go_current_node->is_root( ) eq abap_true or

* 2.) a non-empty element (object, array, member)

      lv_parent_type eq `object` and iv_name is not initial or

    ( lv_parent_type eq `array` or

      lv_parent_type eq `member` ) and iv_name is initial

    ).

endmethod.

As you can see, it also checks for the proper usage of the "name" attribute: It must be used only if the parent element is an <object>.

For closing elements, an optional parameter iv_name can be passed, only for making the client code more robust.

method close_element.

  data: lo_element type ref to if_ixml_element.

* This method does *not* manipulate the DOM tree

* It only changes the current node,

* which is the reference for further insertions

* For named elements: Assert that this is the correct element to be closed

* The parameter iv_name is optional here, it only improves robustness

  if iv_name is not initial.

    lo_element ?= go_current_node.

    assert id zdev condition lo_element->get_attribute( `name` ) eq iv_name.

  endif.

* Go back one level

  go_current_node ?= go_current_node->get_parent( ).

  eo_me = me.

endmethod.

For the same purpose - robustness - there are dedicated methods for closing arrays, objects and members, which additionally ensure that the correct element has been closed. See here, for instance, the method close_array( ):

method close_array.

  assert id zdev condition go_current_node->get_name( ) eq `array`.

  close_element( iv_name ).

  eo_me = me.

endmethod.

Getting the Result

As already mentioned, ultimately a cl_sxml_string_writer instance will be necessary for producing the JSON string. In our class, the method get_json() does this final job of transforming the iXML document with the identity transformation into the string writer, and to return the resulting output (which will be an UTF-8 string, ideal for passing it over into a web application).

At this stage, the stack should be resolved: All elements which had been opened during the build process, have to be closed properly. This is equivalent to the condition that the current node now points to the root element again.

method get_json.

* Nesting must be resolved properly

  assert id zdev

    condition go_current_node->is_root( ) eq abap_true.

* Debugging: If ZDEV is switched on, you may inspect the object in method debug_result( )

  assert id zdev

    condition debug_result( ) eq abap_true.

  data: lo_writer type ref to cl_sxml_string_writer.

  lo_writer = cl_sxml_string_writer=>create( if_sxml=>co_xt_json ).

  call transformation id

    source xml go_document

    result xml lo_writer.

  ev_result = lo_writer->get_output( ).

endmethod.

Inspecting the iXML Document in the Debugger

Sometimes, we would like to check the produced IF_IXML_DOCUMENT object in the debugger at this point. Since the debugger does not provide a built-in view for the iXML object, we have to transform it into an XSTRING if we want to see it as XML document. For efficiency, we only want this to be performed while developing.

As can be seen above, I use a checkpoint group again for this purpose: Only if the checkpoint group is active (as per our SAAB settings: only for development systems), the method debug_result( ) will be called. If the checkpoint group is inactive, the method call will be skipped completely. With a breakpoint, we can branch into the method and inspect the resulting lv_xml byte string with the XML data view:

method debug_result.

  data: lv_xml type xstring,

        lo_parser type ref to if_ixml_parser.

* Checking well-formedness by transforming the document into an xstring

  try.

      call transformation id

        source xml go_document

        result xml lv_xml.

      ev_transformation_done = abap_true.

    catch cx_root.

  endtry.

endmethod.

Conclusion

Usually, a JSON result can be built at once from given data structures, using an XSLT or ST transformation which renders into a cl_sxml_string_writer. In the (more exceptional) cases where a JSON-XML document is to be built piece by piece, a helper class is a useful choice, having an iXML document as inner state.

Under this link

http://bsp.mits.ch/code/clas/zcl_json_output

you can find the full source code of my writer class zcl_json_output.

Its unit test section should document all of its features - including those not presented in this blog.