payment-card-util

Java tools for payment card systems: ISO 8583 messages, Mastercard IPM clearing and parameter files, card numbers, pin blocks and key handling.

This is a port of the Python cardutil package by Anthony Delosa. It is checked against cardutil 0.7.3: the tests replay files and messages generated by the Python package and compare byte for byte. See Differences from cardutil for the few places the two disagree on purpose.

Requires Java 21 or later.

Modules

Module	Artifact	Dependencies
core	com.sgerrand:payment-card-util	none
cli	com.sgerrand:payment-card-util-cli	picocli, Jackson

The library has no runtime dependencies. Only the command line tools pull anything in.

<dependency>
  <groupId>com.sgerrand</groupId>
  <artifactId>payment-card-util</artifactId>
  <version>0.1.2-SNAPSHOT</version>
</dependency>

Packages

Package	What it holds
card	Card numbers, BINs, schemes, Luhn check digits, masking
iso8583	Reading and writing ISO 8583 messages, including PDS, DE 43 and chip data
ipm	Mastercard IPM clearing files, VBS framing, 1014 byte blocking, parameter extracts
pin	ISO 9564 format 0 and 4 pin blocks, Visa pin verification values
crypto	Key components, key check values, key encryption
config	The message and file layouts everything else works from

Reading a clearing file

try (InputStream in = Files.newInputStream(path);
     IpmReader reader = IpmReader.blocked(in)) {
    for (Iso8583Message message : reader) {
        message.mti();          // 1240
        message.text(2);        // the card number
        message.number(4);      // the amount, as a long
        message.dateTime(12);   // the local transaction time
        message.pds(158);       // a Mastercard private subelement
        message.iccTag("9F02"); // a chip tag out of DE 55
    }
}

If nobody recorded how the file was written, ask it:

IpmInfo info = IpmInfo.inspect(Files.newInputStream(path));
info.valid();     // does this look like an IPM file at all
info.blocked();   // 1014 byte blocking
info.encoding();  // ASCII, EBCDIC or UNKNOWN
info.charset();   // a character set of that kind, to read the file with

Writing one

try (OutputStream out = Files.newOutputStream(path);
     IpmWriter writer = IpmWriter.blocked(out)) {
    writer.write(Iso8583Message.builder()
            .mti("1240")
            .de(2, "4444555566667777")
            .de(4, 12345L)
            .de(12, LocalDateTime.now())
            .pds(158, "0000000000")
            .build());
}

Closing the writer adds the end of file marker and fills out the last block. A file left unclosed has neither.

Messages on their own

Iso8583Message message = Iso8583.parse(bytes);
byte[] out = Iso8583.serialize(message);

For a file in EBCDIC, a hex bitmap, or a layout of your own:

Iso8583Options options = Iso8583Options.defaults()
        .withCharset(Iso8583Options.EBCDIC_CP500)
        .withConfig(myConfig);
Iso8583Message message = Iso8583.parse(bytes, options);

Card numbers

Pan pan = Pan.parse("4111 1111 1111 1111");
pan.isLuhnValid();  // true
pan.scheme();       // CardScheme.VISA
pan.bin();          // 411111
pan.toString();     // 411111******1111

Pan.toString() always masks, so logging one by accident cannot leak the full number. Call pan.digits() where you need the real value.

Pin blocks

Iso0PinBlock block = new Iso0PinBlock("1234", "1111222233334444");
block.toBytes();                // the clear block
block.toEncryptedBytes(ppk);    // encrypted under a pin protection key
block.toPvv(pvvKey);            // the Visa verification value

Iso0PinBlock.fromEncryptedBytes(bytes, cardNumber, ppk).pin();

Command line tools

Every release has the tools attached as a single jar. Download payment-card-util-cli-<version>-all.jar from the latest release, along with the .sha256 file beside it if you want to check the download:

java -jar payment-card-util-cli-0.1.2-SNAPSHOT --help

The jar carries everything it needs, so nothing else has to be installed. To build it from source instead:

mvn package
java -jar cli/target/payment-card-util-cli-*-all.jar --help

Command	What it does
mci-ipm-to-csv	Write the messages in an IPM file out as CSV
mci-csv-to-ipm	Build an IPM file from CSV
mci-ipm-encode	Rewrite an IPM file in another character set or format
mci-ipm-param-to-csv	Write one parameter table out as CSV
mci-ipm-param-encode	Rewrite a parameter file in another character set or format

java -jar payment-card-util-cli-0.1.2-SNAPSHOT mci-ipm-to-csv clearing.ipm
java -jar payment-card-util-cli-0.1.2-SNAPSHOT mci-ipm-encode clearing.ipm \
    --in-encoding cp500 --out-encoding latin_1 --out-format VBS

A layout other than the built in one goes in a JSON file, in the same shape cardutil uses:

java -jar payment-card-util-cli-0.1.2-SNAPSHOT mci-ipm-to-csv clearing.ipm \
    --config-file my-layout.json

The CARDUTIL_CONFIG environment variable also works: point it at a directory holding cardutil.json.

A layout that packs something this library has never seen inside an element needs a FieldCodec rather than a change to the parser. Name the processor in the config file:

{"bit_config": {"43": {"field_name": "Card acceptor", "field_type": "LLVAR",
                       "field_length": 0, "field_processor": "BRANCH-CODE"}}}

then register a codec under that name:

Iso8583Options options = Iso8583Options.defaults()
        .withConfig(myLayout)
        .withCodec("BRANCH-CODE", (bit, raw, text, field) -> Map.of(
                "DE" + bit + "_BRANCH", text.substring(0, 4)));

The name is yours to choose. FieldProcessors holds the five cardutil knows, and the reader has no list of its own to add to: it asks the settings which codec a field's name maps to. A name nothing has a codec for leaves the field as it stands, which is what cardutil does with a processor it does not recognise.

A config file says what is different, not what the whole layout is. Naming one data element changes that element and leaves the rest of the built in layout alone, and the same goes for a parameter table. The exception is output_data_elements, a column order rather than a set of parts, so naming it replaces the list.

Differences from cardutil

Everything else matches byte for byte. These do not, and each is deliberate.

• mci-ipm-to-csv masks card numbers. cardutil writes them in full. Pass --unmask-pan for the same output as the Python tool. A column counts as holding a card number if the layout gives it the PAN or PAN-PREFIX field processor, or names it PAN. Either is enough: the built-in Mastercard layout marks no element by processor, since it comes from cardutil's own config, and a layout that does mark one may still name another.
• The PAN and PAN-PREFIX field processors only mark a field. cardutil masks the value and cuts the prefix short while parsing. That loses what the file actually held, so the message cannot be written back unchanged and a caller who needs the real number cannot ask for it. Here they say which field holds a card number and nothing more: Iso8583.parse returns what the file holds, and masking is left to whatever shows the data.
• IpmInfo.inspect detects blocking properly. cardutil only reports blocking for files of exactly one or two blocks, so it answers "not blocked" for almost every real file.
• Pin length is written as a hex digit. For pins of 10 to 12 digits cardutil writes it as decimal text, which shifts the rest of the block along and produces a block that cannot be read back. Pins of 9 digits or fewer are identical.
• Private data too long to fit raises an error. cardutil silently drops a subelement of 993 characters or more.
• A message must have a message type indicator to be written. cardutil writes a record without one, and then cannot read that record back: its own reader answers "Failed decoding MTI field". The type indicator sits in front of the bitmap, so a missing or short one shifts the rest of the record along. Writing it out is refused rather than producing a file nothing can read.
• --in-format and --out-format take VBS or BLOCKED_1014, where cardutil takes vbs or 1014. Every tool also takes --no1014blocking, as cardutil's do; on the encoders it covers both files and wins over the format options.
• mci-ipm-param-encode takes --config-file, which cardutil's does not. Records are copied as they stand, so the only thing read from the file is how long a record may be before the file is called damaged. Without it that limit could only be moved by the CARDUTIL_CONFIG environment variable, while every other tool could be pointed at a file.

Proving it against cardutil

The parity tests read core/src/test/resources/vectors/cardutil.json, which is generated by running the Python package. tools/gen_config.py regenerates DefaultConfig.java, the message layout, the same way. Run both from the repository root:

python -m venv .venv
.venv/bin/pip install 'cardutil[crypto]'
.venv/bin/python tools/gen_config.py \
    core/src/main/java/com/sgerrand/paymentcardutil/config/DefaultConfig.java
.venv/bin/python tools/gen_vectors.py \
    core/src/test/resources/vectors/cardutil.json

The root holds no cardutil directory, so the installed package is the one that gets imported. Neither generator runs during the build; both are run by hand when tracking a new cardutil release.

Because those files are committed, they only speak for the cardutil release they came from. The cardutil drift workflow runs weekly, installs the newest cardutil, regenerates both files, runs the tests against them and fails if either file moved. When the weekly run does not succeed it opens an issue, assigned to the repository owner, that says which case below applies. A later run that still fails rewrites that issue rather than opening another.

Read the result like this:

Drift	Tests	What it means
no	pass	Nothing to do.
yes	pass	cardutil changed something cosmetic. Regenerate and commit.
yes	fail	cardutil changed behaviour. The port needs updating to match.
no	fail	Not the new release. The tests fail against what is committed.

The table only holds when every check ran. A check that was skipped or cancelled has not passed, and a generator that stopped half way leaves files that moved and tests that never ran. Read the log before regenerating anything.

It also only holds for a new release. If the newest cardutil is still the one the committed files came from, nothing has shipped, and whatever failed will fail on pull requests too.

The failing run keeps the regenerated files as an artifact, so the new output can be read without installing anything.

The command line is hand written, so regenerating proves nothing about it. The same workflow reads cardutil's own argument parsers and this port's --help, and fails if a tool has grown an option we lack, or has one of ours that the README does not list as a divergence. Run it the same way:

mvn package
python tools/check_cli_options.py

It exits 1 when the options differ: add the option, or list it above as a divergence and in EXPECTED_EXTRA in that script. It exits 2 when it could not finish comparing, which says nothing about the options either way.

Build

mvn verify

The Java version lives in .tool-versions, which asdf and mise read, and which the workflows point at so that local and CI builds agree. The test matrix is the exception: it names its versions, since running on more than one is the point.

Formatting is google-java-format in its four space form, applied by Spotless. mvn verify fails on anything out of shape; mvn spotless:apply puts it back. The generated DefaultConfig.java is left alone, since its generator decides how it looks.

SpotBugs runs over the compiled classes in the same build, at its highest effort and lowest reporting threshold. What it is told to ignore lives in config/spotbugs-exclude.xml, each entry with a reason; nothing is waved through without one.

Reading and writing messages is all these tools do, and a clearing file holds millions of records, so speed there is worth checking rather than believing. ParseWriteBenchmark times both against one clearing message. Nothing runs it during the build:

mvn -pl core test-compile
java -cp core/target/classes:core/target/test-classes \
    com.sgerrand.paymentcardutil.iso8583.ParseWriteBenchmark

To measure a change, build the version to compare against in a worktree and run the same class against each; the class comment says how. It is a stopwatch rather than a laboratory: enough to see a few per cent, not enough to argue about one.

Releasing

Releases are cut by release-please. A push to main keeps a release pull request up to date from the commit messages. Merging it tags the version, writes CHANGELOG.md, sets the version in every pom.xml and publishes a GitHub Release.

Publishing that release sends the library to Maven Central. Only payment-card-util and the parent it inherits from are published; the command line tools are a download from the GitHub Release, since the shaded jar bundles its dependencies and is meant to be run rather than depended on.

Nothing is held back for approval, and a version on Maven Central can never be replaced or withdrawn, so the publish workflow refuses to run if the project version does not match the release tag, or if it is still a snapshot.

Four repository secrets are needed, on top of the two the release workflow uses:

Secret	What it is
MAVEN_CENTRAL_USERNAME	User token name from the Central Portal
MAVEN_CENTRAL_TOKEN	The matching token
MAVEN_GPG_PRIVATE_KEY	Armoured private key used to sign artifacts
MAVEN_GPG_PASSPHRASE	Its passphrase

The com.sgerrand group also has to be verified on the Central Portal, which is done once by adding a DNS TXT record to sgerrand.com.

Handling card data

Test data in this repository uses published test card numbers only. Never commit a real card number, and never log a full PAN. .gitignore blocks *.ipm and testdata/ so a real file cannot be committed by accident.

Why the cryptography looks wrong

Scanners flag DesKeys, and they are right about the algorithms: Triple DES is old, and ECB is a poor way to encrypt anything of any length. Both are here on purpose.

Pin blocks and payment keys are defined in terms of these algorithms. ISO 9564 format 0 blocks are encrypted with Triple DES, format 4 blocks with AES, key check values are a block of zeroes encrypted under the key, and a Visa PVV is the first four digits of a Triple DES result. A library that used AES-GCM instead would produce values no payment system would accept, and would fail every parity test against cardutil.

ECB's usual weakness is that identical plaintext blocks give identical ciphertext, which leaks the shape of a long message. Everything here encrypts a single fixed size block: an eight or sixteen byte pin block, or a key. There is no second block for a pattern to show up in.

So com.sgerrand.paymentcardutil.crypto is not a general purpose encryption toolkit, and should not be borrowed as one. Two CodeQL alerts covering these lines are dismissed as "won't fix" for the reasons above; if a scanner reports them again, this is the answer.

Licence

BSD 2-Clause. See LICENSE.

The ported parts carry Anthony Delosa's MIT licence, reproduced in NOTICE.

Mastercard is a registered trademark of Mastercard International Incorporated.