DHL (Germany) without user-provided API-Key

[relishdumpling]

Hi! My First PR here :P

As discussed in #81 this approach tries to implement the open API endpoint that DHL uses for their tracking sites in Germany (#81 (comment)). The International Tracking sits behind a challenge instead.

What changed:

• Refactored DHL Delivery Service -> DHL International Delivery Service.
• Added DHL Germany Delivery Service (with mentioned approach).
  • Also tried to implement the overwrite of shipping Events if only local delivery is happening (See this Comment). Surely this can still be discussed :).

Problems:

• It seems like this endpoints often rejects connections coming from a VPN. This causes unreliability (e.g. background updates) and is just annoying. I still didn't figured out how and if even able to bypass.
• Weird Endpoint (german json objects but with english string locale), also should not rely on everything given, as returned values differ on every shipment.
• Also sudden changes are expected if DHL decides to protect that endpoint even more than just a right user-agent or vpn detection (if this even is the case). Still better option for now than creating a user account with a api-key tbh.

Additions:

• It looks like that the mentioned Deutsche Post in the Issue Deutsche Post support #64 also uses a similar API (Request & Returns) as DHL.
  Endpoint for Deutsche Post Tracking Page: https://www.deutschepost.de/int-verfolgen/data/search?piececode=00340435099029344393&language=en. So I think it's just DHL under the hood (As Deutsche Post also belongs to the "DHL Group").

[itsvic-dev]

here's a preliminary review of the PR so far.
once the PR is ready, i will review again

[itsvic-dev]

please use null instead of a hard-coded English string

[relishdumpling]

It looks like you can't null the parameter 'location' and need to use "" instead. For example InPostItaly works about the same with: item.agency ?: "".

[relishdumpling]

Will work on a vpn fix soon.. if someone else also got any idea please comment

edit:
What I already tried (and didn't work):

• set a "realistic" user-agent header
• general realistic headers (Accept-*, Sec-Fetch-*, Cache-Control, DNT, Referer/Origin, ...)

What I still need to try:

• CookieJar (Is this wanted?)
• Any other solution (calling api endpoint in a browser on phone with the same vpn server works tho, so I hope there is any other solution than webview, because we don't want that :P)

[relishdumpling]

Bad News: quick update on the VPN thing, might be wrong here but:

looked a bit closer at the response headers/cookies and it seems to be a Akamai Bot Manager (_abck, ak_bmsc, bm_mi cookies show up). if that's really it.

tried so far, doesn't seem to help (if my theory above is right):

• realistic UA + realistic headers (Accept-, Sec-Fetch-, Cache-Control, DNT, Referer/Origin)
• CookieJar
• tweaking ConnectionSpec/cipher suites (didn't know what to do here but has something to do with TLS? - still won't work)

not 100% sure on any of this tbh, just going off the cookie names. if someone knows more about Akamai bot detection feel free to comment :)

if it is Akamai though, don't think this is fixable with plain OkHttp/Retrofit realistically in my current understanding. An Option would be webview (which we don't want)...

tl:dr:

fetching dhl (germany) with vpn isn't working most of the time, probably because of a akamai bot manager. Tried some bypasses but didn't work. Probably issue will persist until the use of webviews.

maybe easiest to just document it as a known limitation for now? (But idk how many people regularly use vpn or not tho)

(that's really disappointing mhpff)