JSON and Wish List

[giovannicoppola]

Very nice package! 💯 Is there a way to retrieve all the library details in JSON format? I can convert the .tsv file to JSON, but I was wondering if it is possible to do that directly.

Second, is there a way to download the Audible Wish List?
Thanks.

[mkb79]

Very nice package! 💯 Is there a way to retrieve all the library details in JSON format? I can convert the .tsv file to JSON, but I was wondering if it is possible to do that directly.

The Audible server response is JSON. So it is easy to output this data directly!

Second, is there a way to download the Audible Wish List? Thanks.

I can request the wishlist data from the Audible API. However, in order to download a file, it must first be in your own library.

[giovannicoppola]

that would be great! I would like to create an Alfred workflow to quickly query my library and wishlist, something that is typically cumbersome on the website

[mkb79]

@giovannicoppola
The development branch can now export your library as tsv, csv or json. In the next step I will implement the wishlist.

[giovannicoppola]

@mkb79 I was wondering if the wishlist is still on your roadmap?

[mkb79]

@giovannicoppola I‘ve implement the wishlist in the current development branch. You can use it when you clone it. Or you must wait until I release v0.1.0 to Pypi.

[giovannicoppola]

awesome! I will give it a try

[mkb79]

Here you can find the changelog for the development branch!

[giovannicoppola]

I don't know enough to use the devel branch ☹️, I will wait for the pip release. thanks!

[mkb79]

Simply use git clone --branch development https://github.com/mkb79/audible-cli! Then switch into the cloned dir and run pip install --upgrade .!

[giovannicoppola]

awesome!!! I got the wishlist.

How do I get the json output? can one have additional fields, or is it the exact same as tsv? figured it out 🤓

[giovannicoppola]

thanks! I might need some hand holding to use the python module directly. right now I just launch the CLI version

[mkb79]

I can add a command to audible-cli which let you use the API directly such as wget goes! So you don’t have to do with authentication/different marketplaces and so on. Instead of a url you provide the method (GET,POST,…), the endpoint, the params of the request and a body on POST requests and it gives you the the JSON response!

[giovannicoppola]

or, if you have a couple of examples searching the database using the package and urllib I think I can adapt and implement.. this is my (low) level of expertise

[mkb79]

If you want to use urllib you have to implement the authentication and authorization part yourself. This can be very tricky.

[giovannicoppola]

no I was thinking to leave everything as is now (authentication via CLI), but just search and add to wishlist using urllib. It should work if you are already authenticated, right?

[mkb79]

Ah, you mean using urllib to make a request to audible-cli which make the request to the API and gives you the response? I can running a Flask Server (with Rest Implantation) via a subcommand. Then you can make requests to the Flask server! But this need more time!

[giovannicoppola]

that works! for some reason on my other computer audible library list works from everywhere while on this 'fresh' computer I need to enter python3 -m audible_cli library list. Sorry for the noob questions!

[mkb79]

Which OS you are using. You can search for a file named audible or audible.exe (depending on your OS).

[giovannicoppola]

macOS big Sur (the main computer where everything works is Monterey). I can launch from the terminal but not from Alfred (I get these errors below), I will try to figure it out. It could be a conflict between Python3 installations. Typically it is solved by adding
export PATH=/opt/homebrew/bin:/usr/local/bin:$PATH but no luck so far

[14:27:48.076] ERROR: alfred-audible[[Script Filter](alfredpreferences:workflows%3Eworkflow%3Egiovanni-audible%3E0AC8F1DB-D086-42D0-8052-72A969F42ACD)] Code 1: Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/runpy.py", line 185, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/runpy.py", line 144, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/Users/giovanni.coppola/Library/Python/3.8/lib/python/site-packages/audible_cli/__init__.py", line 5, in <module>
    from .cli import main, quickstart
  File "/Users/giovanni.coppola/Library/Python/3.8/lib/python/site-packages/audible_cli/cli.py", line 6, in <module>
    from .cmds import build_in_cmds, cmd_quickstart
  File "/Users/giovanni.coppola/Library/Python/3.8/lib/python/site-packages/audible_cli/cmds/__init__.py", line 3, in <module>
    from . import (
  File "/Users/giovanni.coppola/Library/Python/3.8/lib/python/site-packages/audible_cli/cmds/cmd_activation_bytes.py", line 2, in <module>
    from audible.activation_bytes import (
ModuleNotFoundError: No module named 'audible.activation_bytes'; 'audible' is not a package

[mkb79]

Maybe you can use Pythons venv module to solve this issue?

[giovannicoppola]

update: in the working computer (number 1) audible is in /usr/local/bin; in the second computer it is in /System/Volumes/Data/Users/giovanni.coppola/Library/Python/3.8/bin/ and it works if I launch from there (both from terminal and Alfred). I tried a third computer fairly new (OS Monterey) and it fails while installing Pillow

[mkb79]

@giovannicoppola
I've released the api_release branch. You can test this out. It contains a proof-of-concept without any exception handling. Please tell me, if this is something you can work with. The Flask-Plugin will come in some days.

To use it this way:

• audible api is the command name (maybe I change this?)
• each query parameter must be provided with the -p/--param option, like - p response_groups=product_desc,media,sample. Yo can chain them with multiple -p options like -p response_groups=media -p num_results=5.
• for POST or PUT requests you can provide the body (as json) with the -b option
• at least you have to provide the endpoint like library, wishlist or catalog/products

Feel free to report me issues, if you find some, or feature requests.

[giovannicoppola]

this is beautiful @mkb79. I am able to browse the catalog and retrieve the options you listed. a couple of questions:

1. I see that the total number of records is 65,422. I see there is a 50 download limit, but any chance the whole database (or at least title, author, narrator, asin etc) can be downloaded every month or so for fast querying? After all this can only be result in more business for Audible...
2. how do I search? if you have an example I can go from there.
3. unrelated: have you ever looked into doing something similar for libby? That would also benefit from a better interface/search...

[mkb79]

@giovannicoppola

1. You can use the page param to retrieve more items. But one user reports to me, that after some pages (I have 40 in my mind?!) the fetched items are always the same. So there will be a limit for doing this.
2. To search you can use the keyword=… parameter. Or if you know what you searching for you can use the title, author or narrator paramaters. Maybe you can mix them too?!
3. I have many ideas, what I would do with this project. A better interface are one of them. But I'm working alone on this project. So I can’t realize all my ideas. But help is welcome every time!

[mkb79]

I‘ve added the api command to the development branch. So this will be released with v0.1.0!

[giovannicoppola]

they all work except keyword?

audible api catalog/products -p num_results=1 -p response_groups=product_desc                     
{'product_filters': [], 'products': [{'asin': '1524779261', 'subtitle': 'An Easy & Proven Way to Build Good Habits & Break Bad Ones', 'title': 'Atomic Habits'}], 'response_groups': ['product_desc', 'always-returned'], 'total_results': 64954}

audible api catalog/products -p num_results=1 -p response_groups=product_desc -p keyword=churchill 
{'product_filters': [], 'products': [{'asin': '1524779261', 'subtitle': 'An Easy & Proven Way to Build Good Habits & Break Bad Ones', 'title': 'Atomic Habits'}], 'response_groups': ['product_desc', 'always-returned'], 'total_results': 64954}

[mkb79]

@giovannicoppola You have to use keywords instead of keyword ;)

[giovannicoppola]

duh, thanks. and is there an endpoint that returns also author, narrator, and the little summary?

[mkb79]

You have to provide the correct response_group to get this info.

Known response_groups values for the catalog/product endpoint are: contributors, customer_rights, media, price, product_attrs, product_desc, product_extended_attrs, product_plan_details, product_plans, rating, sample, sku, series, reviews, ws4v, relationships, review_attrs, categories, category_ladders, claim_code_url, in_wishlist, listening_status, periodicals, provided_review, product_details! You can select multiple response_groups and separate them with a comma!

[mkb79]

Here is my first attempt for a FlaskServer

import logging

import click
from audible import Client
from flask import Flask
from flask.cli import FlaskGroup, ScriptInfo
from flask_restful import Api, Resource, request


logger = logging.getLogger("audible_cli.cmds.cmd_flask")

audible_cli_session = None
audible_cli_client = None


class RESTapp(Resource):

    @staticmethod
    def get(path=""):
        args = request.args
        r = audible_cli_client._request("GET", path, params=args)
        return r

    @staticmethod
    def post(path=""):
        json_body = request.get_json()
        r = audible_cli_client._request("POST", path, json=json_body)
        return r

    @staticmethod
    def put(path=""):
        json_body = request.get_json()
        r = audible_cli_client._request("PUT", path, json=json_body)
        return r

    @staticmethod
    def delete(path=""):
        r = audible_cli_client._request("DELETE", path)
        return r


def create_app():
    app = Flask(__name__)
    api = Api(app)
    api.add_resource(RESTapp, '/', '/<path:path>')
    return app


@click.group(
    "flask-server",
    cls=FlaskGroup,
    create_app=create_app,
    load_dotenv=False
)
@click.pass_context
def cli(ctx):
    """Using Flask to make request to the API"""

    # There can only be one `obj` in ctx. FlaskGroup needs his ScriptInfo
    # The audible_cli.session obj must be keep for further purposes
    global audible_cli_session
    global audible_cli_client
    audible_cli_session = ctx.obj
    audible_cli_client = Client(auth=audible_cli_session.auth)
    obj = ScriptInfo(
        create_app=create_app,
        set_debug_flag=ctx.command.set_debug_flag
    )
    ctx.obj = obj

Copy this to file named cmd_flask-server.py in the audible-cli plugin dir. Then install flask-restful in your venv or something else with pip install flask-restful. You can start it with audible flask-server run --host 0.0.0.0. It support the full Flask run command with his parameters!

[giovannicoppola]

hi @mkb79 thanks for this. I am now able (with audible api) to query the catalog and return audiobook details, which is neat!! I haven't tried but I understand I should be able to add them to my wishlist as well.

Is the flask server needed to alleviate the inconsistencies in installation I am observing across computers?

[mkb79]

The flask-server command works like the api command. The advantage of the flask-server command is, you can run them on one computer (with Python installed) and use a second computer to request the AudibleAPI using the Flask server. You can use a browser or any other APP, which makes http requests, for this. This reduce dependencies for the second computer.

[giovannicoppola]

catalog search demo. you can't set sorting criteria, correct?

[mkb79]

You can try the sort_by param like sort_by: [-Author, -DateAdded, -Price, -Rating, -Title, Author, DateAdded, Price, Rating, Title]

[mkb79]

Audible API is non public. So there is no official documentation about available endpoints, params and message bodys! The Audible app for Windows, Android or iOS uses them. So all information are obtained via HTTP sniffing!

[giovannicoppola]

got it thanks! did you mean `-p sort_by:Author`? That raises an `Uncaught Exception`

…

On Tue, Mar 8, 2022 at 1:58 PM mkb79 ***@***.***> wrote: Audible API is non public. So there is no official documentation about available endpoints, params and message bodys! The Audible app for Windows, Android or iOS uses them. So all information are obtained via HTTP sniffing! — Reply to this email directly, view it on GitHub <#57 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABMBYS4HCJ3A54FSLIWHRYTU66PMTANCNFSM5MZNZJKA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[mkb79]

You have to use a = instead of : as separator. The catalog/products endpoint don’t have a sort_by param if found out now. But you can use products_sort_by: [-ReleaseDate, ContentLevel, -Title, AmazonEnglish, AvgRating, BestSellers, -RuntimeLength, ReleaseDate, ProductSiteLaunchDate, -ContentLevel, Title, Relevance, RuntimeLength]

[giovannicoppola]

Hi @mkb79 I am preparing the documentation and instructions to install alfred-audible. In general, alfred workflow developers try to streamline the process as much as possible, reduce the burden on the end-user, take care of dependancies etc. I can't figure out what I did on my desktop that added audible-cli to the path, so I will point my script to ~/Library/Python/3.8/bin/audible. Is this the right way to do it or is there some system shortcut that points automatically to the most recent Python installation? Will users need to update as they change Python version? Thank you for your patience and guidance! As you can tell I am a beginner. 🤓

[giovannicoppola]

hello @mkb79 I see you released version 0.1.0!! I will proceed as I outlined in the post above, unless you have a different suggestion. This topic was recently discussed in a thread on the Alfred forum and the recommendation is to bundle everything, however I don't know how to launch audible-cli at the command line without installing it. If you agree I will just ask users to install and then I will access using path/to/python

[mkb79]

hello @mkb79 I see you released version 0.1.0!! I will proceed as I outlined in the post above, unless you have a different suggestion. This topic was recently discussed in a thread on the Alfred forum and the recommendation is to bundle everything, however I don't know how to launch audible-cli at the command line without installing it. If you agree I will just ask users to install and then I will access using path/to/python

Maybe you can try the standalone binary for audible-cli. This should bring you all you need in one file (if you do not use your own plugins)!

[giovannicoppola]

ooh, do you now have a macos standalone? The link seems dead

[mkb79]

Instead using the open command please try ./audible --version from the directory where the binary is.

[giovannicoppola]

./audible works normally, thanks. Version is 0.1.0. Not extensive testing, but it seems considerably slower

[mkb79]

The binary have to be extracted every time, before it can be executed. On linux this takes only some milliseconds extra time. On Windows machines it depends on the Anti-Virus-Scanner and some other things. Maybe it‘s the same problem on macOS!

As a solution for Windows, I've added a onedir in addition to the onefile binary. This reduce startup time extremely. I will provide such a onedir solution for macOS in the next release, too.

[giovannicoppola]

ok, happy to help you troubleshoot. this is the output of time:
./audible wishlist list 0.57s user 0.15s system 10% cpu 7.216 total

[mkb79]

Thank you for this information. I‘ve added the onedir binary creation for macOS! The build process will first done automatically on my next release. Then you can test the speed again.

In v0.1.0 you find the new api command. With these you can make custom requests to the AudibleAPI and it will give you the plain JSON output. So you can build your own API requests.

[giovannicoppola]

Hi @mkb79 , I hope you are well! I am back at this project and would like to release a first version of the workflow. I would like to bundle your package with my workflow (in the lib directory) with this command (requirements.txt = audible-cli:

pip3 install -r requirements.txt --target=lib

but I get the following error:
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. httplib2 0.19.1 requires pyparsing<3,>=2.4.2, but you have pyparsing 3.0.8 which is incompatible.

any recommendations?

[mkb79]

Hi @giovannicoppola me is fine, I hope you too!

The only dependencies in my project which requires pyparsing is the packaging package. This need pyparsing>=2.0.2,!=3.0.5! So you can add pyparsing<3,>=2.4.2 to your requirement.txt! This should solve your issue!

[mkb79]

FYI I saw this warning, not sure if it comes from my code or the library

Thank you for your information. That’s my code which gives you the SyntaxWarning.

I am testing on a different computer and it seems I did not bundle everything? Or it's because it is an M1 mac (I get a lot of mach-o, but wrong architecture error messages when I try to run audible-quickstart. I might create 2 builds, one for older Macs and one for M1s.

Sorry I can’t help you by this issue. I don’t have a M1!

[giovannicoppola]

Unfortunately while everything works on my computer, but I can't get to install on a fresh computer, now I get permission issues

[giovannicoppola]

/lib/PIL/_imaging.cpython-38-darwin.so' not valid for use in process: library load disallowed by system policy)

[giovannicoppola]

OK, I think I can use this solution for now: require the user to install pip3 and install everything else via pip3 the first time the app runs. this seems to work.

[mkb79]

OK, I think I can use this solution for now: require the user to install pip3 and install everything else via pip3 the first time the app runs. this seems to work.

Yeah, I think it’s the best solution when the user install the packages itself. Or you use Pyinstaller to build a standalone binary and deliver this with your project.