Repository navigation
Mach-O: Load bundles (MH_BUNDLE) and resolve their imports - #891
Open
NopeCharon wants to merge 1 commit into
Open
NopeCharon wants to merge 1 commit into
NopeCharon wants to merge 1 commit into
Conversation
A bundle is the image dlopen loads as a plugin, and the format Python extension modules use on macOS. The Mach-O backend refused it with "Unsupported Mach-O file type: 8". A bundle is laid out like a dylib and only lacks an install name, so it now takes the dylib paths: it is position independent, placed at 0 when it is the main object, and placed by the loader when it is loaded as a dependency. A bundle's imports also often name no library. With -bundle_loader they are bound to the main executable, and with -undefined dynamic_lookup they are bound by flat lookup. The backend did not handle either: - An nlist stores the main executable as ordinal 0xFF. Chained fixup imports store the same 8-bit value, so they reused that symbol, and its library_name indexed past the end of imported_libraries. Loading such a bundle next to its host raised an IndexError. - Bind opcodes store the special ordinals as negative numbers. A BindingSymbol only checked for 0xFE, so -1 and -2 indexed imported_libraries from the end and quietly named the last library. Chained fixup ordinals are now read as dyld reads them, with the top 16 values of the field taken as negative BIND_SPECIAL_DYLIB_* values. Those constants now carry their mach-o/loader.h values. Every Mach-O symbol class now has binds_to_main_executable and binds_by_flat_lookup, and reports no library name for a special ordinal. MachOSymbolRelocation resolves the first kind against the loader's main object and the second against every candidate image in load order, using exported symbols only. Each falls back to the extern object as before. An import with no library name is no longer matched against an object whose binary_basename is None. The tests use the bundles from angr/binaries#279: aarch64 with chained fixups, x86_64 with bind opcodes, and an x86_64 flat-lookup bundle. Each is loaded both on its own and into its host executable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #444.
Test fixtures: angr/binaries#279. It needs to be merged first. CI checks it out from the reference here.
Problem
A Mach-O bundle (
MH_BUNDLE) is the imagedlopenloads as a plugin, and the format Python extension modules use on macOS. cle refuses to load one:Accepting the file type alone is not enough. A bundle's imports often name no library: with
-bundle_loaderthey are bound to the main executable, and with-undefined dynamic_lookupthey are bound by flat lookup. The backend handles neither:0xFF. A chained-fixup import stores the same 8-bit value, so it reuses that symbol, andSymbolTableSymbol.library_nameindexes past the end ofimported_libraries. Loading a bundle next to its host raisesIndexError: list index out of rangeinMachOSymbolRelocation.resolve_symbol.BIND_OPCODE_SET_DYLIB_SPECIAL_IMM).BindingSymbol.library_nameonly checks for0xFE, so-1and-2indeximported_librariesfrom the end and quietly name the last library.Changes
MH_BUNDLEtakes theMH_DYLIBpaths. It is position independent, placed at 0 when it is the main object, and placed by the loader when it is loaded as a dependency.BIND_SPECIAL_DYLIB_*values. Those constants insymbol.pynow carry theirmach-o/loader.hvalues (-1,-2,-3) instead of0xFF,0xFE,0xFD.binds_to_main_executableandbinds_by_flat_lookup, andlibrary_namereturnsNonefor a special ordinal instead of indexingimported_librarieswith it.MachOSymbolRelocation.resolve_symbolresolves a main-executable import against the loader's main object, and a flat-lookup import against every candidate image in load order (main object first). It only uses exported symbols. If nothing provides the symbol, it falls back to the extern object as before. An import with no library name is no longer matched against an object whosebinary_basenameisNone, such as one loaded from a stream.Weak lookup (
-3) still resolves to the extern object, as it did before.Testing
The new tests in
tests/test_macho.pyuse the fixtures from angr/binaries#279:test_bundleloads the aarch64 bundle (chained fixups) and the x86_64 bundle (bind opcodes) on their own.test_bundle_binds_to_bundle_loaderloads each into its host executable. It checks that the-bundle_loaderimports resolve to the host's exports and that the GOT slot holds the host's variable, whileprintfstill goes through libSystem.test_bundle_flat_lookupdoes the same for the-undefined dynamic_lookupbundle.All three fail on master. The rest of
tests/test_macho*.pyandtests/test_universal2.pypass. On 10 existing Mach-O fixtures in angr/binaries, every relocation resolves to the same target with the same value as on master, for both path and stream loading.