Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions custom-recipes/buildernet/mkosi/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.flashbots-images/
.runtime/
106 changes: 106 additions & 0 deletions custom-recipes/buildernet/mkosi/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# BuilderNet VM Scripts

Temporary bash scripts for running BuilderNet VM alongside builder-playground.
Feel free to translate them to Go and integrate into the CLI.

## Information:

Changes are made in two repos:
- [flashbots-images](https://github.com/flashbots/flashbots-images/tree/fryd/mkosi-playground) on `fryd/mkosi-playground` branch,
- [builder-playground](https://github.com/flashbots/builder-playground/tree/fryd/mkosi-playground) on `fryd/mkosi-playground` branch,

There are no plans of using [buildernet-playground](https://github.com/flashbots/buildernet-playground) repo.

## Start playground

Start playground with: `--bind-external` (expose ports to the VM) and `--contender` (create transactions)

```bash
builder-playground \
start buildernet \
--bind-external \
--output ".playground-home" \
--contender \
--detached fryd-vm-builder
```

## First Time Setup

```bash
./sync.sh # Clone / fetch flashbots-images repo
./build.sh # Build VM image with mkosi
./prepare.sh # Extract image + create data disk
```

`sync.sh` clones/updates the `fryd/mkosi-playground` branch of flashbots-images.

## Run VM

```bash
./start.sh # Start VM (background)
./console.sh # Open console to the VM
./ssh.sh # SSH into running VM (requires SSH key setup)
./stop.sh # Stop VM
```

## Builder Hub

```bash
./builderhub-configure.sh # Register VM with builder-hub and update config for the VM
./builderhub-get-config.sh # Get configuration for the VM
```

## Operator API

Scripts to interact with the operator-api service running inside the VM.

> **Note:** Actions and File Uploads could potentially be used for various things, like injecting genesis config instead of BuilderHub - still exploring this functionality.

```bash
./operator-api-health.sh # Check if operator-api is healthy
./operator-api-logs.sh # Get event logs
./operator-api-action.sh <action> # Execute an action
```

Available actions:
- `reboot` - Reboot the system
- `rbuilder_restart` - Restart rbuilder-operator service
- `rbuilder_stop` - Stop rbuilder-operator service
- `fetch_config` - Fetch config from BuilderHub
- `rbuilder_bidding_restart` - Restart rbuilder-bidding service
- `ssh_stop` - Stop SSH service
- `ssh_start` - Start SSH service
- `haproxy_restart` - Restart HAProxy service

### File Uploads

Upload files to predefined paths. Only whitelisted names from `[file_uploads]` config are allowed:

```toml
[file_uploads]
rbuilder_blocklist = "/var/lib/persistent/rbuilder-operator/rbuilder.blocklist.json"
```

```bash
# Stores local blocklist.json content to the configured remote path
curl -k --data-binary "@blocklist.json" https://localhost:13535/api/v1/file-upload/rbuilder_blocklist
```

### Customization

To add more actions or file uploads, modify the config template:
https://github.com/flashbots/flashbots-images/blob/fryd/mkosi-playground/mkosi.profiles/playground/mkosi.extra/usr/lib/mustache-templates/etc/operator-api/config.toml.mustache

## Maintenance

```bash
./sync.sh # Update flashbots-images to latest
./clean.sh # Clean build artifacts + runtime files
```

## Ports

| Port | Service |
|------|---------|
| 2222 | SSH (maps to VM:40192) |
| 13535 | Operator API (maps to VM:3535) |
14 changes: 14 additions & 0 deletions custom-recipes/buildernet/mkosi/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/usr/bin/env bash
# Build the BuilderNet VM image
set -eu -o pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

FLASHBOTS_IMAGES_DIR="${SCRIPT_DIR}/.flashbots-images"

if [[ ! -d "${FLASHBOTS_IMAGES_DIR}" ]]; then
echo "Error: flashbots-images not found. Run ./sync.sh first."
exit 1
fi

make -C "${FLASHBOTS_IMAGES_DIR}" build-playground
6 changes: 6 additions & 0 deletions custom-recipes/buildernet/mkosi/builderhub-get-config.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
# Check builder-hub configuration
set -eu -o pipefail

echo "Builder Configuration:"
curl -s http://localhost:8888/api/l1-builder/v1/configuration | jq .
21 changes: 21 additions & 0 deletions custom-recipes/buildernet/mkosi/clean.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/usr/bin/env bash
# Clean build artifacts and runtime files
set -eu -o pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

FLASHBOTS_IMAGES_DIR="${SCRIPT_DIR}/.flashbots-images"
RUNTIME_DIR="${SCRIPT_DIR}/.runtime"
PIDFILE="${RUNTIME_DIR}/qemu.pid"

# Check if VM is running
if [[ -f "${PIDFILE}" ]] && kill -0 $(cat "${PIDFILE}") 2>/dev/null; then
echo "Error: VM is still running. Run ./stop.sh first."
exit 1
fi

if [[ -d "${FLASHBOTS_IMAGES_DIR}" ]]; then
make -C "${FLASHBOTS_IMAGES_DIR}" clean
fi

rm -rf "${RUNTIME_DIR}"
18 changes: 18 additions & 0 deletions custom-recipes/buildernet/mkosi/console.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/usr/bin/env bash
# Connect to the BuilderNet VM console (auto-login with devtools profile)
set -eu -o pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
CONSOLE_SOCK="${SCRIPT_DIR}/.runtime/console.sock"

if [[ ! -S "${CONSOLE_SOCK}" ]]; then
echo "Error: Console socket not found. Is the VM running?"
echo "Run ./start.sh first."
exit 1
fi

echo "Connecting to VM console... (Ctrl+] to exit)"
socat -,raw,echo=0,escape=0x1d UNIX-CONNECT:"${CONSOLE_SOCK}"

echo ""
echo "Disconnected from VM console."
25 changes: 25 additions & 0 deletions custom-recipes/buildernet/mkosi/operator-api-action.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/usr/bin/env bash
# Execute an operator-api action
set -eu -o pipefail

OPERATOR_API_PORT=${OPERATOR_API_PORT:-13535}
OPERATOR_API_HOST=${OPERATOR_API_HOST:-localhost}

ACTION=${1:-}

if [ -z "$ACTION" ]; then
echo "Usage: $0 <action>"
echo ""
echo "Available actions:"
echo " reboot - Reboot the system"
echo " rbuilder_restart - Restart rbuilder-operator service"
echo " rbuilder_stop - Stop rbuilder-operator service"
echo " fetch_config - Fetch config from BuilderHub"
echo " rbuilder_bidding_restart - Restart rbuilder-bidding service"
echo " ssh_stop - Stop SSH service"
echo " ssh_start - Start SSH service"
echo " haproxy_restart - Restart HAProxy service"
exit 1
fi

curl -s -k "https://${OPERATOR_API_HOST}:${OPERATOR_API_PORT}/api/v1/actions/${ACTION}"
17 changes: 17 additions & 0 deletions custom-recipes/buildernet/mkosi/operator-api-health.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/usr/bin/env bash
# Check operator-api health
set -eu -o pipefail

OPERATOR_API_PORT=${OPERATOR_API_PORT:-13535}
OPERATOR_API_HOST=${OPERATOR_API_HOST:-localhost}

# /livez returns HTTP 200 with empty body on success
HTTP_CODE=$(curl -s -k -o /dev/null -w "%{http_code}" "https://${OPERATOR_API_HOST}:${OPERATOR_API_PORT}/livez")

if [ "$HTTP_CODE" = "200" ]; then
echo "OK"
exit 0
else
echo "FAIL (HTTP $HTTP_CODE)"
exit 1
fi
8 changes: 8 additions & 0 deletions custom-recipes/buildernet/mkosi/operator-api-logs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
# Get operator-api event logs
set -eu -o pipefail

OPERATOR_API_PORT=${OPERATOR_API_PORT:-13535}
OPERATOR_API_HOST=${OPERATOR_API_HOST:-localhost}

curl -s -k "https://${OPERATOR_API_HOST}:${OPERATOR_API_PORT}/logs"
57 changes: 57 additions & 0 deletions custom-recipes/buildernet/mkosi/ovmf.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
#!/usr/bin/env bash
# Discover OVMF firmware files using QEMU firmware descriptors.
# Usage:
# source ovmf.sh
# # Sets OVMF_CODE and OVMF_VARS variables
set -eu -o pipefail

_discover_ovmf() {
local search_dirs=(
/usr/share/qemu/firmware
/etc/qemu/firmware
)

for dir in "${search_dirs[@]}"; do
[[ -d "$dir" ]] || continue
for f in "$dir"/*.json; do
[[ -f "$f" ]] || continue
# Match x86_64 UEFI, non-secure-boot, 4m variant
if jq -e '
(.targets[] | select(.architecture == "x86_64")) and
(.["interface-types"] | index("uefi")) and
((.features | index("secure-boot")) | not)
' "$f" >/dev/null 2>&1; then
OVMF_CODE=$(jq -r '.mapping.executable.filename' "$f")
OVMF_VARS=$(jq -r '.mapping."nvram-template".filename' "$f")
if [[ -f "$OVMF_CODE" && -f "$OVMF_VARS" ]]; then
return 0
fi
fi
done
done

# Fallback: check common hardcoded paths
local known_paths=(
"/usr/share/OVMF/x64"
"/usr/share/edk2/x64"
"/usr/share/OVMF"
"/usr/share/edk2/ovmf"
"/usr/share/edk2-ovmf/x64"
)
for dir in "${known_paths[@]}"; do
if [[ -f "$dir/OVMF_CODE.4m.fd" && -f "$dir/OVMF_VARS.4m.fd" ]]; then
OVMF_CODE="$dir/OVMF_CODE.4m.fd"
OVMF_VARS="$dir/OVMF_VARS.4m.fd"
return 0
fi
done

echo "Error: Could not find OVMF firmware files." >&2
echo "Install the edk2-ovmf package (or equivalent for your distro)." >&2
return 1
}

_discover_ovmf
export OVMF_CODE OVMF_VARS
echo "OVMF_CODE=${OVMF_CODE}"
echo "OVMF_VARS=${OVMF_VARS}"
14 changes: 14 additions & 0 deletions custom-recipes/buildernet/mkosi/playground.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
base: buildernet
description: Deploy the stack with the BuilderNet mkosi image (QEMU)

recipe:
builder-vm:
services:
builder:
lifecycle_hooks: true
init:
- rm -rf .runtime/buildernet-vm.qcow2 || true
- ./prepare.sh
start: ./start.sh
stop:
- ./stop.sh
30 changes: 30 additions & 0 deletions custom-recipes/buildernet/mkosi/prepare.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#!/usr/bin/env bash
# Extract VM image and create data disk
set -eu -o pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

FLASHBOTS_IMAGES_DIR="${SCRIPT_DIR}/.flashbots-images"
RUNTIME_DIR="${SCRIPT_DIR}/.runtime"

QEMU_QCOW2="${FLASHBOTS_IMAGES_DIR}/mkosi.output/buildernet-qemu_latest.qcow2"

VM_IMAGE="${RUNTIME_DIR}/buildernet-vm.qcow2"
VM_DATA_DISK="${RUNTIME_DIR}/persistent.raw"

if [[ ! -f "${QEMU_QCOW2}" ]]; then
echo "Error: QEMU qcow2 image not found: ${QEMU_QCOW2}"
echo "Run ./build.sh first."
exit 1
fi

rm -rf "${RUNTIME_DIR}"
mkdir -p "${RUNTIME_DIR}"

rm -f "${VM_IMAGE}"
cp --sparse=always "${QEMU_QCOW2}" "${VM_IMAGE}"

qemu-img create -f raw "${VM_DATA_DISK}" 100G

echo "Runtime ready: ${RUNTIME_DIR}"
ls -lah "${RUNTIME_DIR}"
57 changes: 57 additions & 0 deletions custom-recipes/buildernet/mkosi/send-tx.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
#!/usr/bin/env bash

FROM_ADDR="0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266"

CHAIN_ID_HEX=$(curl -s -X POST http://localhost:8545 \
-H "Content-Type: application/json" \
-d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}' | jq -r '.result')
CHAIN_ID=$((CHAIN_ID_HEX))

NONCE_HEX=$(curl -s -X POST http://localhost:8545 \
-H "Content-Type: application/json" \
-d "{\"jsonrpc\":\"2.0\",\"method\":\"eth_getTransactionCount\",\"params\":[\"$FROM_ADDR\",\"pending\"],\"id\":1}" | jq -r '.result')
NONCE=$((NONCE_HEX))

# Tx: Send the builder 0x74085Fbe5108CF75F91951DDfD06d3f7d6890EF7 with 0.1 ether
TX_PAYLOAD=$(cast mktx \
--private-key 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 \
0x74085Fbe5108CF75F91951DDfD06d3f7d6890EF7 \
--value 0.1ether --nonce "$NONCE" --gas-limit 21000 --gas-price 1gwei --chain "$CHAIN_ID")

# Change this to set the target. 8545 for playground reth, 18645 for buildernet vm rbuilder.
date +%H:%M:%S
SEND_RESULT=$(curl -s --fail-with-body -X POST http://localhost:8545 \
-H "Content-Type: application/json" \
-d "{\"jsonrpc\":\"2.0\",\"method\":\"eth_sendRawTransaction\",\"params\":[\"$TX_PAYLOAD\"],\"id\":1}")
CURL_EXIT=$?

if [ $CURL_EXIT -ne 0 ]; then
echo "Error: curl failed with exit code $CURL_EXIT"
exit 1
fi

echo "Send result: $SEND_RESULT"

TX_HASH=$(echo "$SEND_RESULT" | jq -r '.result')

if [ -z "$TX_HASH" ] || [ "$TX_HASH" = "null" ]; then
echo "Error: Failed to get transaction hash"
echo "$SEND_RESULT" | jq .
exit 1
fi

echo "TX_HASH: $TX_HASH"

echo "Waiting for receipt..."
while true; do
RECEIPT=$(curl -s -X POST http://localhost:8545 \
-H "Content-Type: application/json" \
-d "{\"jsonrpc\":\"2.0\",\"method\":\"eth_getTransactionReceipt\",\"params\":[\"$TX_HASH\"],\"id\":1}")

RESULT=$(echo "$RECEIPT" | jq -r '.result')
if [ "$RESULT" != "null" ]; then
echo "Receipt: $RECEIPT"
break
fi
sleep 1
done
7 changes: 7 additions & 0 deletions custom-recipes/buildernet/mkosi/ssh.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/env bash
# SSH into the BuilderNet VM
set -eu -o pipefail

SSH_PORT=2222

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -p ${SSH_PORT} bnet@localhost
Loading
Loading