API post operations

PhilB
Senior
Posts: 224
Joined: Sun May 11, 2014 6:23 pm
Great Britain

API post operations

Post by PhilB » Thu Oct 01, 2015 3:06 pm

I have been slowly chipping away at API documentation even though as a non-developer it seems a bit cheeky.
The APIs which have been documented so far seem to be the GET calls. What is a good way of documenting POST operations and in particular giving an example?

Is this okay? https://www.mythtv.org/wiki/Channel_Ser ... eDBChannel

I'd appreciate views before I do any others.

Phil

User avatar
bill6502
Developer
Posts: 1730
Joined: Fri Feb 07, 2014 5:28 pm
United States of America

Re: API post operations

Post by bill6502 » Thu Oct 01, 2015 7:14 pm

Hi Phil,

Nice work on all your updates to the Wiki.

In the interest of getting a discussion going. My opinion is that the endpoints shouldn't be documented
in the Wiki. Because they'll will be wrong (at least in some cases) when the next version comes out. I prefer
to just look at the output from curl (using the --data option for POSTs.)

If all of the parameters are listed, then someone could reasonably ask what they mean. Requiring
more maintenance of the Wiki e.g. the difference/meaning of StartTime vs. StartTs.

There was some significant discussion on IRC earlier this year (or perhaps last year) about improving the
WSDL for things that currently return decimal values for enumerations. For example in 0.28-pre, one
RecordType is now 'Record One', but used to be 6. A huge improvement in my opinion, but I don't
believe it's available via WSDL yet.

I think the Wiki for the Services API should explain how to get the parameters (e.g. from the WSDL.) For
the record, I disagree with putting the output of mythAnyCommand --help in the Wiki because it's more
up to date from the command line. Another example would be the Python bindings. If a user types:
help (MythBE.getPendingRecordings), they can get the developer's help text which should be up to date
for the working version. Maybe a shorter answer would be: don't document things twice.

There, that should provoke some discussion.

User avatar
dekarl
Developer
Posts: 228
Joined: Thu Feb 06, 2014 11:01 pm
Germany

Re: API post operations

Post by dekarl » Fri Oct 02, 2015 7:01 am

Phil, appreciate your work on the wiki, especially the Perl API Examples. But also agree with Bill, we should not do work twice with our limited resources.

Personally I prefer to extend our automatically generated WSDL documentation with more embedded documentation. That would move the base documentation of the API calls themselves into the code, leaving example use cases for the wiki. Putting the documentation into the WSDL opens up some nice options, like automatically generating a Web Service Client binding in any language with documentation of the service embedded directly into that client binding. (at least that is the idea, though a short research hints that support for wsdl:documentation is sketchy at best)
But the move to enumerations comes with its own set of hurdles to take. E.g. #12510

I do like the idea of providing examples, not of a single service invocation, but instead of common user stories. E.g. how to delete recordings after seven days or how to swap title and subtitle for some recordings.
So documentation for UpdateDbChannel could contain an example of "how to update your channels from a central database to avoid a rescan". Or a variant of MythUpChUk for setting up MythTV and XMLTV with data from Atlas for a standard Freeview HD setup. But the basic service interface description should move to the code, once parameters can be documented there.

Btw, please file a bug for the duplicate entry trap so it has a chance of getting fixed.

PhilB
Senior
Posts: 224
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: API post operations

Post by PhilB » Fri Oct 02, 2015 11:51 am

I hoped this would raise an interesting debate and I've not been disappointed!
I have raised a ticket on the duplicates problem - see https://code.mythtv.org/trac/ticket/12516#ticket
Not sure why the text editor insisted in throwing in those spurious ?'s though.

Will ponder your comments - first thought is that the API interface is a bit too fluid at present.
Am I right in thinking that the wsdl is generated automatically so that if (say) a parameter in the API code is misnamed by the developer (all too easy!) as (say) 'visible' instead of 'Visible' then that will be reflected in the wsdl?

Phil

PhilB
Senior
Posts: 224
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: API post operations

Post by PhilB » Mon Oct 05, 2015 7:55 pm

An interesting debate. I didn’t quite appreciate how far this would go once I started some innocent cutting and pasting! Perhaps a rethink of where we are going is indeed in order.

Thanks for the curl suggestion.

The API interface has lots of potential and I think we will see loads of interesting add-on programs submitted by folks who might not otherwise see themselves as ‘developers’. I include myself there and it's a fascinating concept. It’s to those people that the wiki should be aimed but I see a number of hurdles:

1. I agree that ‘double documentation’ is to be avoided but I think that the documentation should be sufficient to spark the imagination. Whilst a cut/paste of (say) my 0.27.4 output with all my localisation (UK based, combined FE/BE, etc) will never be an accurate reflection of the reader’s system, the examples in the wiki did at least inspire me to think ‘This GetChannelInfoList has a field called ‘visible’. I could hide all those adult and jewellery channels in one go!! Wow!!’.

One way would be to just include a few examples, warn that they may not reflect local reality and give pointers on how to infer that local reality.

I wonder though whether it would be possible though to craft the wiki so that it has links to access a real Mythtv system directly? I thought of a central myth server (Synology has a nice NAS demo like this) but the maintenance and security might be a nightmare. Using the reader’s own server is attractive. One way might be to create cookies to give the local IP addresses for each port (6544 or 6547) then have a central server to redirect to the local addresses found in the cookie. There must be a simpler way though which is browser independent?

2. We really need to describe some of the API calls. Some names are very obvious, but not for example GetDDLineupList. Again, can we assume that if it does not make sense then it isn't interesting (or relevant) to me?

3. Descriptions of the parameters. I did start on a script to read wsdl pages, extract all parameters from all calls and list them giving parameter name, list of routines which use them as input, those which output and possible aliases (eg Visible and visible). I thought it would be a small list to which we could add descriptions. Supplying the latter was the big problem because that cannot be automated and the job mushroomed beyond the effort available.

Can we assume that if the reader does not understand what a parameter represents then using it would be hazardous so they should stay away? Some certainly do need explanation though such as CallSign or ChannelName? StartTS or StartTime?

4. Stability. It is inevitable that there are inconsistencies in parameters this early in the API development cycle. Chan or Channel, ID or Id, Visible or visible etc. The dilemma for one using the API interface is whether future changes by devs will break their application. Likewise with the routines which are changing (rightly) from Get to Post or changing their name to avoid inconsistency.
Is there a policy on this other than a stark warning to application developers (and users of their code)?

So, it would appear that until that utopian day when everyting is automated we need:
Health warnings in the wiki about the above issues
A few examples but say that their local system may vary
Examples to call their own system via wsdl to learn the input parameters (does it/will it say which are optional/mandatory?).
Ditto for the output of Get functions.
Notes of potential traps.
Notes on future trends – where do we snoop to pick up that gossip?
More examples.

On that note, would one of you do me a great favour - make available the output of a sample wsdl page from a 0.28 system for me somewhere (email?). I want to check that my perl routines which parse wsdl are not broken in 0.28.

Is there a central server we could use for the 'redirection', assuming we could find someone with the necessary skills to code it and we agree that this is a good way forward?

Phil

User avatar
bill6502
Developer
Posts: 1730
Joined: Fri Feb 07, 2014 5:28 pm
United States of America

Re: API post operations

Post by bill6502 » Mon Oct 05, 2015 10:23 pm

Here's a pastebin of the 0.28-pre Channel and Myth services. If you want
more, let me know.

http://paste.ubuntu.com/12692001/

Note that Myth/GetSetting has changed in 0.28-pre. It now returns
exactly one setting. GetSettingList does what the earlier version did.
It's just one example of changes users would need to address.

Maybe you asked over on the developer's list if the WSDL is generated
based on the code itself. I wrote #11916 some time ago. It added
a new endpoint called SendKey. I've verified that it's WSDL is 'automatically'
included. Including that it's a POST.

PhilB
Senior
Posts: 224
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: API post operations

Post by PhilB » Tue Oct 06, 2015 4:21 pm

Thanks Bill, but I was after the XML to throw at my code.
Phil

User avatar
bill6502
Developer
Posts: 1730
Joined: Fri Feb 07, 2014 5:28 pm
United States of America

Re: API post operations

Post by bill6502 » Tue Oct 06, 2015 4:41 pm

Sorry. Can you give me an example of what you'd like?
I only see how to get the WDSL in SOAP format
(and never used it myself.) Or would you like a/some
sample endpoints like <BE>:6544/Myth/GetSetting...

PhilB
Senior
Posts: 224
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: API post operations

Post by PhilB » Tue Oct 06, 2015 6:43 pm

Thanks Bill, you are very patient!
I'm after some raw xml output from a wsdl page. I can then cludge my wsdl parsing stuff to read it and check that it still works. eg

curl 192.168.2.99:6544/Channel/wsdl>output.

and post the output file. That would be great.
Many thanks
Phil

User avatar
bill6502
Developer
Posts: 1730
Joined: Fri Feb 07, 2014 5:28 pm
United States of America

Re: API post operations

Post by bill6502 » Tue Oct 06, 2015 9:21 pm


PhilB
Senior
Posts: 224
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: API post operations

Post by PhilB » Wed Oct 07, 2015 7:42 am

Super. Thanks Bill.

PhilB
Senior
Posts: 224
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: API post operations

Post by PhilB » Sun Oct 18, 2015 5:50 pm

Hi Bill,
That 0.28 wsdl example was very useful. Would you also post two more examples please for validation?
http://backend:6544/Channel/GetChannelI ... =1&Count=2
and
http://Backend:6544/Channel/GetChannelInfo?ChanID=1024
Thanks
Phil

User avatar
bill6502
Developer
Posts: 1730
Joined: Fri Feb 07, 2014 5:28 pm
United States of America

Re: API post operations

Post by bill6502 » Sun Oct 18, 2015 6:51 pm

Hi, I put both here: http://paste.ubuntu.com/12849161/

EDITS:

Here's a GetChannelInfoList that I passed through sed
to get just 2 channels:

http://pastebin.com/uMWEXQUg

And another with Details=true and Visible=true

http://pastebin.com/Qg8KWxGs
Last edited by bill6502 on Tue Oct 20, 2015 4:03 pm, edited 2 times in total.

PhilB
Senior
Posts: 224
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: API post operations

Post by PhilB » Mon Oct 19, 2015 6:31 pm

Thanks Bill.
Er ... the GetChannelInfo is fine - parsed fine.
The wsdl needed a code tweak which is all ready to go up on the wiki.
The GetChannelInfoList is very odd though - it might just be the spurious \ before &Count=2. Would you try again please?

curl http://mc0:6544/Channel/GetChannelInfoL ... =1&Count=2

Many thanks

Phil

User avatar
bill6502
Developer
Posts: 1730
Joined: Fri Feb 07, 2014 5:28 pm
United States of America

Re: API post operations

Post by bill6502 » Tue Oct 20, 2015 1:46 am

The \ is required to prevent the shell from spinning off the
command. I updated my previous response with a new
pastebin of it.

Post Reply