Implementing new Minecraft version support in PocketMine-MP

Minecraft often makes changes to its network protocol, including adding new packets, removing old ones, or changing the structure of existing packets. Because of this, PocketMine-MP often needs to be updated to support the latest version of Minecraft.

This page will cover the basic process of analyzing and implementing protocol updates.

Pre-requisites

Basic changes to BedrockProtocol and PocketMine-MP

BedrockProtocol is where most of the manual work needs to be done.

  1. Use protocol_info_generator_objdump.py in BedrockProtocolDumper to generate packet ID lists and version information. This script requires Python 2, and takes a path to a bedrock_server_symbols.debug file and a path to your BedrockProtocol local copy. The following things will be updated:

  • src/PacketPool.php

  • src/ProtocolInfo.php

  • src/PacketHandler.php

  • src/PacketHandlerInterface.php

  • New files may be added if there are new packets in the version you are updating to. However, files will not be removed for deleted packets - that’s up to you to do manually.

  1. Analyze what changes need to be made to packet structures. This typically involves one or more of the following methods:

  • Reading the official Minecraft Bedrock protocol documentation - these are the easiest to use, but sometimes miss changes or are incorrect

  • Using tools like IDA to decompile bedrock_server_symbols.debug and analyze packet source code (you need the x86_64 decompiler to work on BDS)

  1. Update packet structures and information in BedrockProtocol. This includes:

  • Writing code to encode and decode new packets

  • Adjusting structures of existing packets and their subtypes if necessary

  • Updating constants and/or enums to match the new version

  • Whether new packet(s) should be clientbound, serverbound, or both

  1. Run tools/generate-create-static-methods.php in BedrockProtocol. This will update the ::create() methods for all packets to match the new packet structures. (Make sure to run php-cs-fixer afterwards. It won’t break anything if you don’t, but the script will mess up the formatting.)

  2. Do any necessary changes to PocketMine-MP to make it compatible with the updated BedrockProtocol. You can run PHPStan to find out where changes need to be made.

Note

You can link locally-modified versions of BedrockProtocol and related repositories to PocketMine-MP. This makes it easier to do integration testing without having to commit anything.

Use the provided script to link your local copies of BedrockProtocol, BedrockBlockUpgradeSchema, BedrockItemUpgradeSchema, and BedrockData to PocketMine-MP.

Note

If you suspect that the official protocol documentation is incorrect, you can use tools like IDA to decompile the BDS binary and analyze the packet code directly. However, this can be a very frustrating and time-consuming process, as IDA is very slow and laggy when working with large binaries like BDS.

You can also use gdb to look at BDS’s compiled assembly code, which can be much faster, but is also much more difficult to understand.

Generating supporting data

PocketMine-MP requires additional data from the new version of Minecraft to function correctly. This includes:

These data originate from two sources:

  1. Mods of the Bedrock Dedicated Server that dump the needed data. This method is a pain, but it’s the only way to get some data. This could become a problem in the future if Mojang proceed with their plans to remove debugging symbols from the BDS.

  2. Packet traces of the Bedrock Dedicated Server communicating with a vanilla Minecraft client. These can be obtained in several different ways, but not all required data can be obtained this way.

Getting data from BDS via mods

PMMP provides a modding toolkit that makes it easy to run the needed mods on the BDS. Follow the instructions in the repository README to set up the modding environment and run the mods.

The mod code of interest can be found in the data extraction mod main repository. This mod is preinstalled by the modding toolkit when you clone following the instructions in the README.

Once you’ve generated the data, copy all the files (not the folders) in input_files/ to BedrockData. There may be additional files that are not needed by BedrockData. You can ignore these.

Note

The code often needs to be updated to work with the latest version of the BDS. This guide won’t cover how to make the mods run on newest BDS, as the changes needed are usually different from version to version, and this guide would end up very long. You really should have general experience modding BDS before trying to get into this. If you need help, ask in the PMMP Discord server.

Warning

Make sure the input_files/old_block_palettes submodule is up to date, and that it contains a block palette for the previous version of Minecraft. You’ll need this later for generating blockstate upgrade schemas.

Getting data from vanilla <-> vanilla packet traces

The modding toolkit also provides a tracer script that can be used to hook into a running instance of BDS and capture packet traces between a vanilla client and server. This script uses the Frida.re Python API to hook into packet read and write functions in the BDS. The script has no impact on vanilla behaviour, guaranteeing the best quality data.

These traces can be used to generate data, and also to verify that your changes to BedrockProtocol are correct.

Steps to capture packet traces:

  1. Create a new world in Minecraft on the target version. Make sure to enable any experiments which add new blocks or items, as these need to be present for generating data upgrade schemas.

  2. Configure server.properties on your BDS to use the world you generated.

  3. Start bedrock_server_symbols.debug directly (do not use start.sh).

  4. In a separate terminal, run the following command: sudo python3 tracer.py rw bedrock_server_symbols.debug. This will hook into the running BDS instance and start capturing packet traces.

  5. Join the BDS server using Minecraft. Do whatever in-game tests you need to get the game to send packets you want to see.

  6. Stop the server. The script will print the filename of the trace file it generated. This usually looks something like packets_123456789.txt.

Once you have a trace file, use PocketMine’s tools/generate-bedrock-data-from-packets.php script, providing the path to the trace file as an argument and the path to your local copy of BedrockData. The script will update the appropriate files in BedrockData.

Warning

Do not use tracer.py on a server with mods loaded. The BDS instance may crash or behave unexpectedly.

Note

You may have difficulty joining a BDS server running inside WSL2 from a Windows Minecraft client. This is a long-standing issue between WSL2 and UWP apps and has no known fix. You can work around it by using a basic proxy script like RakLib proxy.php and joining via the proxy instead of trying to connect directly. Alternatively, just run the server on a proper Linux machine or VM.

Note

If you don’t want to use tracer.py, you can also create a packet trace using a proxy such as gophertunnel. The structure of the file is simple: each line is starts with read: or write: followed by the packet buffer encoded as base64. However, a proxy may change the structure, order and timings of packets, so it may not give the same quality of data as the tracer script.

Generating a blockstate upgrade schema

PocketMine-MP uses JSON schemas to tell it how to upgrade blockstate NBT data in old world saves to the newest version.

Steps to generate a blockstate upgrade schema:

  1. Get the appropriate palette mapping file that the BDS mod generated. You can find it in mapping_files/old_palette_mappings, and the file name will be something like 1.20.80.24_beta_to_current_block_map.bin.

  2. Use PocketMine-MP’s tools/generate-blockstate-upgrade-schema.php to generate a new schema for this version.

  3. Add the schema to the nbt_upgrade_schema folder of BedrockBlockUpgradeSchema. The name should be prefixed with a number to ensure the files are sorted correctly, like this: 0271_1.20.70.24_beta_to_1.20.80.24_beta.json.

  4. Commit the new schema. Do not commit directly to the master branch until the version is released.

Note

If you can’t get a palette mapping file, you can write an upgrade schema by hand. However, this may be time-consuming and error-prone, and is not recommended unless you have no other choice.

Generating an item upgrade schema

PocketMine-MP uses JSON schemas to upgrade item data in old world saves to the newest version.

Steps to generate an item upgrade schema:

  1. Use PocketMine-MP’s tools/generate-item-upgrade-schema.php. Give it the path to r16_to_current_item_map.json in BedrockData, and the path to the already-existing schemas in BedrockItemUpgradeSchema.

  2. Add the schema to the id_meta_upgrade_schema folder of BedrockItemUpgradeSchema. The name should be prefixed with a number to ensure the files are sorted correctly, like this: 0181_1.20.70.24_beta_to_1.20.80.24_beta.json.

  3. Commit the new schema. Do not commit directly to the master branch until the version is released.

Completing changes in PocketMine-MP using the new data

Once you have generated supporting data, you may need to do a few more changes to PocketMine-MP.

This mostly involves updating the code in src/data/bedrock/block and src/data/bedrock/item to decode and encode data in the expected format for the newest version.

If you’re lucky, the version you’re updating to might not have changed anything at all. In this case, you won’t need to do anything.

Steps to do the changes:

  1. Run composer update-codegen.

  2. Run vendor/bin/phpstan. This will tell you where you need to make changes.

  3. Fix all the problems reported by PHPStan.

  4. Make sure the following constants are updated correctly:

    • BlockStateData::CURRENT_VERSION (often changes)

    • BedrockWorldData::CURRENT_STORAGE_VERSION (rarely changes)

    • BedrockWorldData::CURRENT_STORAGE_NETWORK_VERSION (always changes)

    • BedrockWorldData::CURRENT_CLIENT_VERSION_TARGET (always changes)

    • LevelDB::CURRENT_LEVEL_CHUNK_VERSION (rarely changes)

    • LevelDB::CURRENT_LEVEL_SUBCHUNK_VERSION (rarely changes)

  5. Run vendor/bin/phpunit tests/phpunit. Make sure all the tests pass. If you’ve made a mistake somewhere, the tests should fail.

Playtesting PocketMine-MP

Once you’ve made all the changes, you should playtest PocketMine-MP to make sure everything works as expected.

  1. Create a new world with Minecraft on the target version.

  2. Load the world into PocketMine-MP and start the server.

  3. Do whatever playtests you need to make sure your changes work as expected.

Committing the results

Once you’re happy with your changes, commit the changes on all repositories. By convention, we recommend you name your branch like this: bedrock-1.21.0.

When the new version’s final release is out, merge the changes into the main branches and tag releases where appropriate.

Note

The Bedrock* repositories use tag metadata (suffixed using a + sign) to show which version of Minecraft they support. An example tag looks something like this: 1.9.0+bedrock-1.20.80.

The metadata doesn’t affect dependency version resolution, but it can be useful for telling at a glance which version the repository supports.