#### m1n1 running as hypervisor, over dwc USB and serial

$ dwc python run_guest.py ../build/m1n1.macho
Initializing hypervisor over iodev IODEV.USB0
LOAD: _HDR 16384 bytes from 0 to 0
LOAD: TEXT 98304 bytes from 4000 to 4000
LOAD: RODA 32768 bytes from 1c000 to 1c000
LOAD: DATA 393216 bytes from 24000 to 24000
ZERO: 573440 bytes from 0x84000 to 0x110000
LOAD: PYLD 0 bytes from 84000 to 110000
SKIP: 67108864 bytes from 0x110000 to 0x4110000
Fetching ADT (0x00058000 bytes)...
Total region size: 0x6ec000 bytes
Guest region start: 0x81871c000
Loading kernel image (0x110004 bytes)...
.......
Copying SEPFW (0x5d4000 bytes)...
Adjusting SEPFW address in ADT...
Removing ADT node /arm-io/dart-usb0
Removing ADT node /arm-io/atc-phy0
Removing ADT node /arm-io/usb-drd0
Removing ADT node /device-tree/cpus/cpu1
Removing ADT node /device-tree/cpus/cpu2
Removing ADT node /device-tree/cpus/cpu3
Removing ADT node /device-tree/cpus/cpu4
Removing ADT node /device-tree/cpus/cpu5
Removing ADT node /device-tree/cpus/cpu6
Removing ADT node /device-tree/cpus/cpu7
Pushing ADT (330580 bytes)...
Setting up bootargs...
Disabling other iodevs...
 - IODEV.UART
 - IODEV.FB
 - IODEV.USB1
Jumping to entrypoint at 0x818720800

#### m1n1 boots as a guest at EL1 over the serial console

#### In another terminal:

$ python linux.py ../../linux/arch/arm64/boot/Image.gz ../../linux/arch/arm64/boot/dts/apple/t8103-j274.dtb ../../../initramfs/initramfs.cpio.gz -b 'earlycon console=ttySAC0,1500000 console=tty0 debug'
Base at: 0x81871c000
FB at: 0x9e0df8000
Setting boot args: "earlycon console=ttySAC0,1500000 console=tty0 debug"
Loading 2564255 bytes to 0x824b94000..0x824e0609f...
..........................................................................................................................................................................................................................................................................................................................
Loading DTB to 0x824e060c0...
Kernel_base: 0x825000000
Loading 952755 initramfs bytes to 0x824e10000...
.....................................................................................................................
TTY> Starting secondary CPUs...
TTY> FDT: bootargs = 'earlycon console=ttySAC0,1500000 console=tty0 debug'
TTY> FDT: initrd at 0x824e10000 size 0xe89b3
TTY> FDT: framebuffer@9e0df8000 base 0x9e0df8000 size 0x7e9000
TTY> ADT: 64 bytes of random seed available
TTY> FDT: KASLR seed initialized
TTY> FDT: Passing 64 bytes of random seed
TTY> FDT: DRAM at 0x800000000 size 0x200000000
TTY> FDT: Usable memory is 0x81871c000..0x9db5e0000 (0x1c2ec4000)
TTY> FDT: CPU 1 is not alive, disabling...
TTY> FDT: CPU 2 is not alive, disabling...
TTY> FDT: CPU 3 is not alive, disabling...
TTY> FDT: CPU 4 is not alive, disabling...
TTY> FDT: CPU 5 is not alive, disabling...
TTY> FDT: CPU 6 is not alive, disabling...
TTY> FDT: CPU 7 is not alive, disabling...
TTY> FDT prepared at 0x81ca84000
Uncompressing gz ...
6805512
Decompress OK...
Ready to boot
DAIF: c0
TTY> Preparing to boot kernel at 0x825000000 with fdt at 0x81ca84000
--- TTY mode | Quit: CTRL+] | Menu: CTRL+T ---
Preparing to run next stage at 0x825000000...
MMU: shutting down...
MMU: shutdown successful, clearing caches
USB1: shutdown
Vectoring to next stage...
[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x611f0221]
[    0.000000] Linux version 5.12.0-04309-gffd05156fe5c (marcan@raider) (aarch64-linux-gnu-gcc (Gentoo 10.2.0-r5 p6) 10.2.0, GNU ld (Gentoo 2.35.1 p2) 2.35.1) #7 SMP PREEMPT Tue Apr 27 20:03:23 JST 2021
[    0.000000] random: fast init done
[... snip ...]
[    0.000000] Root IRQ handler: aic_handle_irq
[    0.000000] Root FIQ handler: aic_handle_fiq

### Hang! An exception happened. Back in the hypervisor console...

Guest exception: SYNC
  SPSR = 0xa00000c5 (N=1, Z=0, C=1, V=0, Q=0, IT10=0, DIT=0, SSBS=0, PAN=0, SS=0, IL=0, GE=0x0, IT72=0x0, E=0, A=0, I=1, F=1, T=0, M=0x5(EL1h))
  ELR =  0xffffffd7ee05fb04
  ESR =  0x6232f805 (ISS2=0x0, EC=0x18(MSR), IL=1, ISS=0x32f805)
  FAR =  0x0
    x0-x3 = 0000000000000001 ffffffd7ee05faf0 0000000000000001 0000000000000000
    x4-x7 = 0000000000000000 ffffffa45a6f34a8 00000000ffffffff ffffffd7ee4267a0
   x8-x11 = ffffffd7ee3767a0 fffffffffffc0b50 ffffffd7ee426b90 000000000000002a
  x12-x15 = 000000000000007e ffffffd7ee3767a0 ffffffd86e363bcf 0000000000000048
  x16-x19 = 000000000000000a 000000000000003f fffffffffffc0b50 ffffffd7ee370698
  x20-x23 = 0000000000000000 ffffffd7ee3094a8 0000000000000000 0000000000000001
  x24-x27 = ffffffd7ee36f7c0 ffffffd7ee36f5b8 0000000000000000 0000000002aa55ff
  x28-x30 = 000000000000005f ffffffd7ee363d60 ffffffd7ede32bdc

  == MSR fault decoding ==
  Instruction:   mrs x0, CNTP_CTL_EL0

Entering debug shell
>>>

#### Whoops! Something is configured wrong, and the guest faulted accessing CNTP_CTL_EL0

>>> mrs(CNTHCTL_EL2)
0x400

#### I think I see the problem...

>>> mrs(CNTHCTL_EL2) >> 11
0x0
>>> mrs(CNTHCTL_EL2) >> 10
0x1

#### Both of those bits should be set to properly enable the physical timer at EL1

>>> msr(CNTHCTL_EL2, 3 << 10)
>>> cont

#### Back at the serial terminal...

[    0.000000] irq_apple_aic: Kernel running in EL1, mapping interrupts
[    0.000000] irq_apple_aic: Initialized with 896 IRQs, 4 FIQs, 32 vIPIs
[    0.000000] arch_timer: cp15 timer(s) running at 24.00MHz (virt).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x588fe9dc0, max_idle_ns: 440795202592 ns
[    0.000000] sched_clock: 56 bits at 24MHz, resolution 41ns, wraps every 4398046511097ns
[... snip ...]
[    0.110955] Run /init as init process
[    0.111230]   with arguments:
[    0.111459]     /init
[    0.111633]   with environment:
[    0.111876]     HOME=/
[    0.112056]     TERM=linux
Alive


BusyBox v1.30.1 (Debian 1:1.30.1-6+b1) built-in shell (ash)
Enter 'help' for a list of built-in commands.

/bin/sh: can't access tty; job control turned off
/ # 

#### Win.