Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

starfive visionfive2: use mainline kernel #964

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

onny
Copy link

@onny onny commented Jun 1, 2024

Description of changes

Depends on, which is not yet backported to 24.05 NixOS/nixpkgs#316724

Things done
  • Tested the changes in your own NixOS Configuration
  • Tested the changes end-to-end by using your fork of nixos-hardware and
    importing it via <nixos-hardware> or Flake input

@NickCao
Copy link
Member

NickCao commented Jun 2, 2024

Also please apply:

--- a/starfive/visionfive/v2/default.nix
+++ b/starfive/visionfive/v2/default.nix
@@ -8,8 +8,7 @@
       inherit (config.boot) kernelPatches;
     });

-    kernelParams =
-      lib.mkDefault [ "console=tty0" "console=ttyS0,115200n8" "earlycon=sbi" ];
+    kernelParams = [ "console=tty0" "console=ttyS0,115200n8" "earlycon=sbi" ];

     initrd.availableKernelModules = [ "dw_mmc_starfive" ];

Otherwise these kernelParams are overridden by boot.consoleLogLevel. This is why we were not seeing any output on mainline kernel.

@NickCao
Copy link
Member

NickCao commented Jun 2, 2024

It's working! https://hydra.nichi.co/build/3702

Tested ethernet. USB and PCI-E doesn't seem to work.

screenshot-2024-06-02-17-27-18

@onny
Copy link
Author

onny commented Jun 2, 2024

Current state of the PR is also booting for me :)

@NickCao
Copy link
Member

NickCao commented Jun 2, 2024

Current state of the PR is also booting for me :)

I'd prefer fixing the kernel config once and for all :)

@NickCao
Copy link
Member

NickCao commented Jun 3, 2024

Just made an interesting discovery: u-boot UEFI boot actually works.
screenshot-2024-06-03-14-23-39
So we can theoretically build a generic sdcard image that boots on all riscv boards! As long as they have a on board flash for u-boot.

@0x4A6F
Copy link
Member

0x4A6F commented Jun 3, 2024

This mainline kernel is also running for nearly a day (except for a short power outage due to flooding).
Thanks for the work on this! ❤️

@onny onny force-pushed the starfive2-mainline branch 2 times, most recently from 33e760f to 56039df Compare June 3, 2024 23:25
@onny
Copy link
Author

onny commented Jun 3, 2024

@NickCao incooperated your change requests, still seems to boot fine for me :)

@onny
Copy link
Author

onny commented Jun 3, 2024

need to figure out how to enable USB support

@eshattow
Copy link

eshattow commented Jun 4, 2024

@NickCao There's an efi_loader improvement patch up for review that enables EFI-configured (and U-Boot environment variable as fallback) devicetree loading with EFI boot manager (but not global EFI/BOOT/BOOT{arch}.efi) targets in U-Boot: https://patchwork.ozlabs.org/project/uboot/list/?series=408529 and to your point about a single image for all VisionFive2 variants (Star64, VF2, Mars*) yes although I would want to firstly see U-Boot OF_UPSTREAM adopted for JH7110 targets to de-duplicate the devicetree situation. Booting SPL from SD Card is officially recommended against by StarFive documentation (and is not possible for Mars CM*) so you have to make a choice about what you're going to insist from the end-user. If your users are super nerdy then you have that same problem just deciding about where to cut the line in the sand; Debian devs have said they're on to Grub2 as an intermediate boot loader loaded via EFI and it's least friction to just stick with that. When you upgrade your kernel from OS userland what happens to devicetree files? That's the thing nobody really agrees on is how that process should go. For the VisionFive2 variants there's the common devicetree nodes and then some relatively minor fix-ups on top of that, but at U-Boot phase how is it ever going to know what the user will select in Grub2 for a kernel to load? And conversely, if the devicetree is specified in Grub2 then it's too late at that moment to have applied the runtime fix-ups that allow U-Boot to get to that point of loading Grub2, so there is discontinuity of devicetree nodes going from U-Boot into Linux kernel.

USB host mode broke with the change in devicetree verbiage after Linux Kernel 6.1.x; Slap the devicetree from whatever Linux 5.15 vendor kernel into 6.1.x with an appropriate kernel config and you'll have working USB as a baseline to figure out what changed with the new devicetree verbiage of later kernel releases.

@NickCao
Copy link
Member

NickCao commented Jun 4, 2024

@NickCao There's an efi_loader improvement patch up for review that enables EFI-configured (and U-Boot environment variable as fallback) devicetree loading with EFI boot manager (but not global EFI/BOOT/BOOT{arch}.efi) targets in U-Boot: https://patchwork.ozlabs.org/project/uboot/list/?series=408529

I think this removes the need to keep u-boot updated (for updated dtb)?

and to your point about a single image for all VisionFive2 variants (Star64, VF2, Mars*) yes although I would want to firstly see U-Boot OF_UPSTREAM adopted for JH7110 targets to de-duplicate the devicetree situation. Booting SPL from SD Card is officially recommended against by StarFive documentation (and is not possible for Mars CM*) so you have to make a choice about what you're going to insist from the end-user.

We can have a board specific "firmware updater" and go for the SPL in SPI flash route. And generic-extlinux-compatible already allows use to provide generic images, as long as they are all on a proper build of u-boot. EFI or a chainloaded boot loader is just a nice to have.

If your users are super nerdy then you have that same problem just deciding about where to cut the line in the sand; Debian devs have said they're on to Grub2 as an intermediate boot loader loaded via EFI and it's least friction to just stick with that.

I'm also leaning towards that.

When you upgrade your kernel from OS userland what happens to devicetree files? That's the thing nobody really agrees on is how that process should go. For the VisionFive2 variants there's the common devicetree nodes and then some relatively minor fix-ups on top of that, but at U-Boot phase how is it ever going to know what the user will select in Grub2 for a kernel to load? And conversely, if the devicetree is specified in Grub2 then it's too late at that moment to have applied the runtime fix-ups that allow U-Boot to get to that point of loading Grub2, so there is discontinuity of devicetree nodes going from U-Boot into Linux kernel.

That's a valid concern, I see no good solution to this.

USB host mode broke with the change in devicetree verbiage after Linux Kernel 6.1.x; Slap the devicetree from whatever Linux 5.15 vendor kernel into 6.1.x with an appropriate kernel config and you'll have working USB as a baseline to figure out what changed with the new devicetree verbiage of later kernel releases.

I believe it's working with the 6.6 branch we were using.

@onny
Copy link
Author

onny commented Jun 6, 2024

I added

CONFIG_PHY_STARFIVE_JH7110_PCIE = y
CONFIG_PHY_STARFIVE_JH7110_USB = y

and tried the new kernel, still no output in lspci and lsusb :(

this is the dmesg output
[    0.000000] Linux version 6.9.3 (nixbld@localhost) (riscv64-unknown-linux-gnu-gcc (GCC) 13.2.0, GNU ld (GNU Binutils) 2.41) #1-NixOS4
[    0.000000] Machine model: StarFive VisionFive 2 v1.3B
[    0.000000] SBI specification v1.0 detected
[    0.000000] SBI implementation ID=0x1 Version=0x10003
[    0.000000] SBI TIME extension detected
[    0.000000] SBI IPI extension detected
[    0.000000] SBI RFENCE extension detected
[    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
[    0.000000] Malformed early option 'earlycon'
[    0.000000] efi: UEFI not found.
[    0.000000] OF: reserved mem: 0x0000000040000000..0x000000004001ffff (128 KiB) nomap non-reusable mmode_resv0@40000000
[    0.000000] OF: reserved mem: 0x0000000040020000..0x000000004003ffff (128 KiB) nomap non-reusable mmode_resv1@40020000
[    0.000000] cma: Reserved 16 MiB at 0x00000000ff000000 on node -1
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000023fffffff]
[    0.000000] NUMA: NODE_DATA [mem 0x23fdc1140-0x23fdc2fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000040000000-0x00000000ffffffff]
[    0.000000]   Normal   [mem 0x0000000100000000-0x000000023fffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000040000000-0x000000004003ffff]
[    0.000000]   node   0: [mem 0x0000000040040000-0x000000023fffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000023fffffff]
[    0.000000] SBI HSM extension detected
[    0.000000] CPU with hartid=0 is not available
[    0.000000] riscv: base ISA extensions acdfim
[    0.000000] riscv: ELF capabilities acdfim
[    0.000000] percpu: Embedded 53 pages/cpu s91832 r8192 d117064 u217088
[    0.000000] pcpu-alloc: s91832 r8192 d117064 u217088 alloc=53*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 
[    0.000000] Kernel command line: init=/nix/store/065lbs7nb8hhi2q0z251l639mml3a0xj-nixos-system-nixos-24.11pre-git/init console=tty0 7
[    0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)
[    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Fallback order for Node 0: 0 
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 2064384
[    0.000000] Policy zone: Normal
[    0.000000] mem auto-init: stack:all(zero), heap alloc:off, heap free:off
[    0.000000] software IO TLB: area num 4.
[    0.000000] software IO TLB: mapped [mem 0x00000000f90ba000-0x00000000fd0ba000] (64MB)
[    0.000000] Virtual kernel memory layout:
[    0.000000]       fixmap : 0xffffffc6fea00000 - 0xffffffc6ff000000   (6144 kB)
[    0.000000]       pci io : 0xffffffc6ff000000 - 0xffffffc700000000   (  16 MB)
[    0.000000]      vmemmap : 0xffffffc700000000 - 0xffffffc800000000   (4096 MB)
[    0.000000]      vmalloc : 0xffffffc800000000 - 0xffffffd800000000   (  64 GB)
[    0.000000]      modules : 0xffffffff02aa4000 - 0xffffffff80000000   (2005 MB)
[    0.000000]       lowmem : 0xffffffd800000000 - 0xffffffda00000000   (8192 MB)
[    0.000000]       kernel : 0xffffffff80000000 - 0xffffffffffffffff   (2047 MB)
[    0.000000] Memory: 8074468K/8388608K available (13735K kernel code, 8942K rwdata, 10240K rodata, 9240K init, 901K bss, 297756K rese)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] ftrace: allocating 50157 entries in 196 pages
[    0.000000] ftrace: allocated 196 pages with 3 groups
[    0.000000] trace event string verifier disabled
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4.
[    0.000000] rcu:     RCU debug extended QS entry/exit.
[    0.000000]  Rude variant of Tasks RCU enabled.
[    0.000000]  Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1.
[    0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1.
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 64 local interrupts mapped
[    0.000000] riscv: providing IPIs using SBI IPI extension
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x1d854df40, max_idle_ns: 881590404240 ns
[    0.000001] sched_clock: 64 bits at 4MHz, resolution 250ns, wraps every 2199023255500ns
[    0.000714] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)
[    0.001298] Console: colour dummy device 80x25
[    0.001313] printk: legacy console [tty0] enabled
[    0.004853] Calibrating delay loop (skipped), value calculated using timer frequency.. 8.00 BogoMIPS (lpj=16000)
[    0.004888] pid_max: default: 32768 minimum: 301
[    0.009228] LSM: initializing lsm=capability,landlock,yama
[    0.015759] landlock: Up and running.
[    0.015777] Yama: becoming mindful.
[    0.017397] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.017555] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.024049] CPU node for /cpus/cpu@0 exist but the possible cpu range is :0-3
[    0.026605] riscv: ELF compat mode unsupported
[    0.026616] ASID allocator disabled (0 bits)
[    0.026878] rcu: Hierarchical SRCU implementation.
[    0.026899] rcu:     Max phase no-delay instances is 1000.
[    0.028352] EFI services will not be available.
[    0.029011] smp: Bringing up secondary CPUs ...
[    0.032434] smp: Brought up 1 node, 4 CPUs
[    0.037369] devtmpfs: initialized
[    0.049998] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.050046] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.051946] pinctrl core: initialized pinctrl subsystem
[    0.055604] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.056700] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations
[    0.056875] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.057019] audit: initializing netlink subsys (disabled)
[    0.057315] audit: type=2000 audit(0.056:1): state=initialized audit_enabled=0 res=1
[    0.058202] thermal_sys: Registered thermal governor 'step_wise'
[    0.058299] cpuidle: using governor ladder
[    0.058358] cpuidle: using governor menu
[    0.080417] cpu2: Ratio of byte access time to unaligned word access is 0.01, unaligned accesses are slow
[    0.080418] cpu1: Ratio of byte access time to unaligned word access is 0.01, unaligned accesses are slow
[    0.080418] cpu3: Ratio of byte access time to unaligned word access is 0.01, unaligned accesses are slow
[    0.104514] cpu0: Ratio of byte access time to unaligned word access is 0.01, unaligned accesses are slow
[    0.112415] platform soc: Fixed dependency cycle(s) with /soc/interrupt-controller@c000000
[    0.113381] platform soc: Fixed dependency cycle(s) with /soc/interrupt-controller@c000000
[    0.124368] platform 19800000.csi: Fixed dependency cycle(s) with /soc/isp@19840000
[    0.125479] platform 19800000.csi: Fixed dependency cycle(s) with /soc/isp@19840000
[    0.125587] platform 19840000.isp: Fixed dependency cycle(s) with /soc/csi@19800000
[    0.129543] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[    0.129569] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
[    0.129589] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[    0.129606] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
[    0.131160] Demotion targets for Node 0: null
[    0.137719] fbcon: Taking over console
[    0.137800] ACPI: Interpreter disabled.
[    0.138140] iommu: Default domain type: Translated
[    0.138161] iommu: DMA domain TLB invalidation policy: strict mode
[    0.139908] SCSI subsystem initialized
[    0.140221] libata version 3.00 loaded.
[    0.140522] usbcore: registered new interface driver usbfs
[    0.140634] usbcore: registered new interface driver hub
[    0.140696] usbcore: registered new device driver usb
[    0.141904] Advanced Linux Sound Architecture Driver Initialized.
[    0.143709] vgaarb: loaded
[    0.144425] clocksource: Switched to clocksource riscv_clocksource
[    1.109481] VFS: Disk quotas dquot_6.6.0
[    1.109985] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    1.110548] netfs: FS-Cache loaded
[    1.111071] pnp: PnP ACPI: disabled
[    1.127570] NET: Registered PF_INET protocol family
[    1.128898] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    1.215205] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 131072 bytes, linear)
[    1.215392] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    1.215464] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    1.216225] TCP bind hash table entries: 65536 (order: 10, 4194304 bytes, linear)
[    1.222559] TCP: Hash tables configured (established 65536 bind 65536)
[    1.224277] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, linear)
[    1.224970] UDP hash table entries: 4096 (order: 6, 393216 bytes, linear)
[    1.225518] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, linear)
[    1.226924] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    1.228334] RPC: Registered named UNIX socket transport module.
[    1.228361] RPC: Registered udp transport module.
[    1.228376] RPC: Registered tcp transport module.
[    1.228391] RPC: Registered tcp-with-tls transport module.
[    1.228406] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    1.228476] NET: Registered PF_XDP protocol family
[    1.228511] PCI: CLS 0 bytes, default 64
[    1.230642] Initialise system trusted keyrings
[    1.231097] Trying to unpack rootfs image as initramfs...
[    1.238319] workingset: timestamp_bits=44 max_order=21 bucket_order=0
[    1.256032] NFS: Registering the id_resolver key type
[    1.256156] Key type id_resolver registered
[    1.256174] Key type id_legacy registered
[    1.256369] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    1.256391] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[    1.256751] 9p: Installing v9fs 9p2000 file system support
[    1.390970] NET: Registered PF_ALG protocol family
[    1.391025] Key type asymmetric registered
[    1.391043] Asymmetric key parser 'x509' registered
[    1.391540] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 244)
[    1.391924] io scheduler mq-deadline registered
[    1.391950] io scheduler kyber registered
[    1.398045] riscv-plic c000000.interrupt-controller: mapped 136 interrupts with 4 handlers for 9 contexts.
[    1.401837] SBI CPPC extension NOT detected!!
[    1.487203] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    1.490543] SuperH (H)SCI(F) driver initialized
[    1.505566] loop: module loaded
[    1.509901] usbcore: registered new interface driver uas
[    1.509973] usbcore: registered new interface driver usb-storage
[    1.510174] usbcore: registered new interface driver usbserial_generic
[    1.510227] usbserial: USB Serial support registered for generic
[    1.510513] mousedev: PS/2 mouse device common for all mice
[    1.513604] sdhci: Secure Digital Host Controller Interface driver
[    1.513630] sdhci: Copyright(c) Pierre Ossman
[    1.513864] Synopsys Designware Multimedia Card Interface Driver
[    1.514336] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.514716] hid: raw HID events driver (C) Jiri Kosina
[    1.514863] usbcore: registered new interface driver usbhid
[    1.514882] usbhid: USB HID core driver
[    1.515227] riscv-pmu-sbi: SBI PMU extension is available
[    1.515270] riscv-pmu-sbi: 16 firmware and 4 hardware counters
[    1.515289] riscv-pmu-sbi: Perf sampling/filtering is not supported as sscof extension is not available
[    1.522654] drop_monitor: Initializing network drop monitor service
[    1.525623] NET: Registered PF_INET6 protocol family
[    3.684196] Freeing initrd memory: 22880K
[    3.686178] Segment Routing with IPv6
[    3.686300] In-situ OAM (IOAM) with IPv6
[    3.686418] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    3.688095] NET: Registered PF_PACKET protocol family
[    3.688905] 9pnet: Installing 9P2000 support
[    3.689026] Key type dns_resolver registered
[    3.689122] start plist test
[    3.693972] end plist test
[    3.714651] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level
[    3.715126] registered taskstats version 1
[    3.715625] Loading compiled-in X.509 certificates
[    3.756657] debug_vm_pgtable: [debug_vm_pgtable         ]: Validating architecture page table helpers
[    3.758679] Key type .fscrypt registered
[    3.758705] Key type fscrypt-provisioning registered
[    3.793111] gpio gpiochip0: Static allocation of GPIO base is deprecated, use dynamic allocation.
[    3.793523] starfive-jh7110-sys-pinctrl 13040000.pinctrl: StarFive GPIO chip registered 64 GPIOs
[    3.796141] printk: legacy console [ttyS0] disabled
[    3.816850] 10000000.serial: ttyS0 at MMIO 0x10000000 (irq = 14, base_baud = 1500000) is a 16550A
[    3.816956] printk: legacy console [ttyS0] enabled
[    5.011222] dwmmc_starfive 16020000.mmc: IDMAC supports 32-bit address mode.
[    5.013138] hid_bpf: error while preloading HID BPF dispatcher: -22
[    5.018398] dwmmc_starfive 16020000.mmc: Using internal DMA controller.
[    5.019114] clk: Disabling unused clocks
[    5.024695] dwmmc_starfive 16020000.mmc: Version ID is 290a
[    5.024781] dwmmc_starfive 16020000.mmc: DW MMC controller at irq 15,32 bit host data width,32 deep fifo
[    5.031603] PM: genpd: Disabling unused power domains
[    5.035468] mmc_host mmc1: card is polling.
[    5.040915] ALSA device list:
[    5.062704]   No soundcards found.
[    5.256459] mmc_host mmc1: Bus speed (slot 0) = 45511111Hz (slot req 400000Hz, actual 399220HZ div = 57)
[    5.490714] Freeing unused kernel image (initmem) memory: 9240K
[    5.496791] Run /init as init process
[    5.500484]   with arguments:
[    5.500494]     /init
[    5.500503]   with environment:
[    5.500511]     HOME=/
[    5.500519]     TERM=linux
[    5.504959] mmc_host mmc1: Bus speed (slot 0) = 45511111Hz (slot req 50000000Hz, actual 45511111HZ div = 0)
[    5.514880] mmc1: new high speed SDHC card at address 0007
[    5.522309] mmcblk1: mmc1:0007 MB33A 27.8 GiB
[    5.542179]  mmcblk1: p1 p2 p3
[    5.643196] stage-1-init: [Thu Jan  1 00:00:05 UTC 1970] loading module dm_mod...
[    5.761914] Kernel module BTF mismatch detected, BTF debug info may be unavailable for some modules
[    5.773213] device-mapper: ioctl: 4.48.0-ioctl (2023-03-01) initialised: [email protected]
[    5.789075] stage-1-init: [Thu Jan  1 00:00:05 UTC 1970] running udev...
[    5.854347] stage-1-init: [Thu Jan  1 00:00:05 UTC 1970] Starting systemd-udevd version 255.6
[    6.920674] stage-1-init: [Thu Jan  1 00:00:06 UTC 1970] kbd_mode: KDSKBMODE: Inappropriate ioctl for device
[    6.928455] stage-1-init: [Thu Jan  1 00:00:06 UTC 1970] starting device mapper and LVM...
[    8.268461] random: crng init done
[    8.445481] stage-1-init: [Thu Jan  1 00:00:08 UTC 1970] checking /dev/disk/by-label/NIXOS_SD...
[    8.450123] stage-1-init: [Thu Jan  1 00:00:08 UTC 1970] fsck (busybox 1.36.1)
[    8.455040] stage-1-init: [Thu Jan  1 00:00:08 UTC 1970] [fsck.ext4 (1) -- /mnt-root/] fsck.ext4 -a /dev/disk/by-label/NIXOS_SD
[    8.466889] stage-1-init: [Thu Jan  1 00:00:08 UTC 1970] NIXOS_SD: clean, 138320/1794704 files, 773043/7297019 blocks
[    8.484996] stage-1-init: [Thu Jan  1 00:00:08 UTC 1970] mounting /dev/disk/by-label/NIXOS_SD on /...
[    8.503954] EXT4-fs (mmcblk1p3): mounted filesystem 44444444-4444-4444-8888-888888888888 r/w with ordered data mode. Quota mode: non.
[    8.552472] EXT4-fs (mmcblk1p3): re-mounted 44444444-4444-4444-8888-888888888888 r/w. Quota mode: none.
[    9.568495] EXT4-fs (mmcblk1p3): re-mounted 44444444-4444-4444-8888-888888888888 r/w. Quota mode: none.
[    9.579610] booting system configuration /nix/store/065lbs7nb8hhi2q0z251l639mml3a0xj-nixos-system-nixos-24.11pre-git
[    9.817413] stage-2-init: running activation script...
[   10.979527] stage-2-init: setting up /etc...
[   12.447880] systemd[1]: System time before build time, advancing clock.
[   12.626900] systemd[1]: systemd 255.6 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +O)
[   12.658706] systemd[1]: Detected architecture riscv64.
[   12.682706] systemd[1]: Hostname set to <nixos>.
[   13.258636] systemd[1]: bpf-lsm: BPF LSM hook not enabled in the kernel, BPF LSM not supported
[   14.175847] systemd[1]: Queued start job for default target Multi-User System.
[   14.267110] systemd[1]: Created slice Slice /system/getty.
[   14.291449] systemd[1]: Created slice Slice /system/modprobe.
[   14.315357] systemd[1]: Created slice Slice /system/serial-getty.
[   14.339098] systemd[1]: Created slice User and Session Slice.
[   14.360913] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
[   14.388819] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[   14.412628] systemd[1]: Expecting device /dev/ttyS0...
[   14.432584] systemd[1]: Reached target Local Encrypted Volumes.
[   14.452634] systemd[1]: Reached target Containers.
[   14.472737] systemd[1]: Reached target Path Units.
[   14.492610] systemd[1]: Reached target Remote File Systems.
[   14.512594] systemd[1]: Reached target Slice Units.
[   14.532644] systemd[1]: Reached target Swaps.
[   14.565254] systemd[1]: Listening on Process Core Dump Socket.
[   14.589292] systemd[1]: Listening on Journal Socket (/dev/log).
[   14.613261] systemd[1]: Listening on Journal Socket.
[   14.633426] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.
[   14.659388] systemd[1]: Listening on udev Control Socket.
[   14.681194] systemd[1]: Listening on udev Kernel Socket.
[   14.724825] systemd[1]: Mounting Huge Pages File System...
[   14.750417] systemd[1]: Mounting POSIX Message Queue File System...
[   14.800837] systemd[1]: Mounting Kernel Debug File System...
[   14.813284] systemd[1]: Starting Create List of Static Device Nodes...
[   14.842843] systemd[1]: Starting Load Kernel Module configfs...
[   14.870920] systemd[1]: Starting Load Kernel Module drm...
[   14.899151] systemd[1]: Starting Load Kernel Module efi_pstore...
[   14.927064] systemd[1]: Starting Load Kernel Module fuse...
[   14.954453] systemd[1]: Starting mount-pstore.service...
[   14.983048] systemd[1]: Starting Create SUID/SGID Wrappers...
[   15.004986] systemd[1]: File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!.
[   15.027907] systemd[1]: Starting Journal Service...
[   15.074308] platform cpufreq-dt: deferred probe pending: (reason unknown)
[   15.081211] platform 10200000.phy: deferred probe pending: jh7110-usb-phy: Failed to get app 125m clock
[   15.090679] platform 17020000.pinctrl: deferred probe pending: starfive-jh7110-aon-pinctrl: could not get reset
[   15.090693] platform 16050000.dma-controller: deferred probe pending: (reason unknown)
[   15.090703] platform 16010000.mmc: deferred probe pending: platform: wait for supplier /soc/i2c@12050000/pmic@36/regulators/aldo4
[   15.113293] fuse: init (API version 7.40)
[   15.133080] systemd[1]: Starting Load Kernel Modules...
[   15.160195] systemd[1]: Starting Remount Root and Kernel File Systems...
[   15.191981] systemd[1]: Starting Coldplug All udev Devices...
[   15.226886] systemd[1]: Mounted Huge Pages File System.
[   15.240954] systemd[1]: Mounted POSIX Message Queue File System.
[   15.254492] systemd-journald[376]: Collecting audit messages is disabled.
[   15.266351] systemd[1]: Mounted Kernel Debug File System.
[   15.285264] EXT4-fs (mmcblk1p3): re-mounted 44444444-4444-4444-8888-888888888888 r/w. Quota mode: none.
[   15.297818] systemd[1]: Finished Create List of Static Device Nodes.
[   15.335644] systemd[1]: [email protected]: Deactivated successfully.
[   15.350894] systemd[1]: Finished Load Kernel Module configfs.
[   15.377567] systemd[1]: Started Journal Service.
[   15.473981] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if yo.
[   15.629067] tun: Universal TUN/TAP device driver, 1.6
[   15.791728] systemd-journald[376]: Received client request to flush runtime journal.
[   15.821028] systemd-journald[376]: File /var/log/journal/7e748c70ef4e4574aaaa42bac6e4bb1e/system.journal corrupted or uncleanly shut.
[   17.579279] missing module BTF, cannot register kfunc
[   17.584441] missing module BTF, cannot register kfunc
[   17.884253] Error: Driver 'jh7110-pcie-phy' is already registered, aborting...
[   17.913771] Error: Driver 'jh7110-usb-phy' is already registered, aborting...
[   18.119056] starfive-jh7110-sys-pinctrl 13040000.pinctrl: pin GPIO61 already requested by 100e0000.i2s; cannot claim for 10090000.tdm
[   18.131125] starfive-jh7110-sys-pinctrl 13040000.pinctrl: error -EINVAL: pin-61 (10090000.tdm)
[   18.139802] starfive-jh7110-sys-pinctrl 13040000.pinctrl: error -EINVAL: could not request pin 61 (GPIO61) from group tdm-0.rx-pins l
[   18.154095] jh7110-tdm 10090000.tdm: Error applying setting, reverse things back
[   18.437150] designware-i2s 100e0000.i2s: probe with driver designware-i2s failed with error -110
[   18.482172] pps_core: LinuxPPS API ver. 1 registered
[   18.487203] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]>
[   18.828175] designware-i2s 120b0000.i2s: probe with driver designware-i2s failed with error -110
[   18.880482] jh7110-trng 1600c000.rng: error -ETIMEDOUT: Failed to register hwrng
[   18.887943] jh7110-trng 1600c000.rng: probe with driver jh7110-trng failed with error -110
[   18.904447] dwmmc_starfive 16020000.mmc: Unexpected interrupt latency
[   19.212966] designware-i2s 120c0000.i2s: probe with driver designware-i2s failed with error -110
[   19.214067] dw_axi_dmac_platform 16050000.dma-controller: DesignWare AXI DMA Controller, 4 channels
[   19.273518] gpio gpiochip1: Static allocation of GPIO base is deprecated, use dynamic allocation.
[   19.275194] PTP clock support registered
[   19.301191] starfive-jh7110-aon-pinctrl 17020000.pinctrl: StarFive GPIO chip registered 4 GPIOs
[   19.313561] ssp-pl022 10060000.spi: ARM PL022 driver, device ID: 0x00041022
[   19.332989] ssp-pl022 10060000.spi: mapped registers from 0x0000000010060000 to 0000000005cf0825
[   19.356559] 4 fixed-partitions partitions found on MTD device 13010000.spi.0
[   19.388803] starfive-dphy-rx 19820000.phy: supply mipi_0p9 not found, using dummy regulator
[   19.424537] Creating 4 MTD partitions on "13010000.spi.0":
[   19.468543] 0x000000000000-0x000000040000 : "spl"
[   19.515518] 0x0000000f0000-0x000000100000 : "uboot-env"
[   19.534702] 0x000000100000-0x000000500000 : "uboot"
[   19.551203] 0x000000600000-0x000001000000 : "reserved-data"
[   19.567509] mc: Linux media interface: v0.10
[   19.887928] axp20x-i2c 5-0036: AXP20x variant AXP15060 found
[   19.953001] axp20x-i2c 5-0036: AXP20X driver loaded
[   19.970826] dwmmc_starfive 16010000.mmc: IDMAC supports 32-bit address mode.
[   19.970888] starfive-dwmac 16030000.ethernet: IRQ sfty not found
[   19.978025] dwmmc_starfive 16010000.mmc: Using internal DMA controller.
[   19.990770] dwmmc_starfive 16010000.mmc: Version ID is 290a
[   19.998689] dwmmc_starfive 16010000.mmc: DW MMC controller at irq 42,32 bit host data width,32 deep fifo
[   20.001717] starfive-dwmac 16030000.ethernet: User ID: 0x41, Synopsys ID: 0x52
[   20.010686] cdns-usb3 10100000.usb: Incorrect DRD configuration
[   20.015564] starfive-dwmac 16030000.ethernet:        DWMAC4/5
[   20.025710] cdns-usb3 10100000.usb: probe with driver cdns-usb3 failed with error -22
[   20.026755] starfive-dwmac 16030000.ethernet: DMA HW capability register supported
[   20.042255] starfive-dwmac 16030000.ethernet: RX Checksum Offload Engine supported
[   20.049870] starfive-dwmac 16030000.ethernet: Wake-Up On Lan supported
[   20.056625] starfive-dwmac 16030000.ethernet: TSO supported
[   20.062332] starfive-dwmac 16030000.ethernet: Enable RX Mitigation via HW Watchdog Timer
[   20.073218] starfive-dwmac 16030000.ethernet: Enabled L3L4 Flow TC (entries=1)
[   20.085409] starfive-dwmac 16030000.ethernet: Enabled RFS Flow TC (entries=10)
[   20.093085] starfive-dwmac 16030000.ethernet: TSO feature enabled
[   20.099267] starfive-dwmac 16030000.ethernet: Using 40/40 bits DMA host/device width
[   20.100874] videodev: Linux video capture interface: v2.00
[   20.142064] cpufreq: cpufreq_online: CPU0: Running at unlisted initial frequency: 999999 KHz, changing to: 1500000 KHz
[   20.151949] starfive-dwmac 16040000.ethernet: IRQ sfty not found
[   20.156573] dwmmc_starfive 16010000.mmc: IDMAC supports 32-bit address mode.
[   20.166106] starfive-dwmac 16040000.ethernet: User ID: 0x41, Synopsys ID: 0x52
[   20.168473] dwmmc_starfive 16010000.mmc: Using internal DMA controller.
[   20.175810] starfive-dwmac 16040000.ethernet:        DWMAC4/5
[   20.180034] dwmmc_starfive 16010000.mmc: Version ID is 290a
[   20.185253] starfive-dwmac 16040000.ethernet: DMA HW capability register supported
[   20.190897] dwmmc_starfive 16010000.mmc: DW MMC controller at irq 42,32 bit host data width,32 deep fifo
[   20.198415] starfive-dwmac 16040000.ethernet: RX Checksum Offload Engine supported
[   20.198424] starfive-dwmac 16040000.ethernet: Wake-Up On Lan supported
[   20.222730] starfive-dwmac 16040000.ethernet: TSO supported
[   20.228463] starfive-dwmac 16040000.ethernet: Enable RX Mitigation via HW Watchdog Timer
[   20.233091] mmc_host mmc0: card is non-removable.
[   20.236597] starfive-dwmac 16040000.ethernet: Enabled L3L4 Flow TC (entries=1)
[   20.250072] starfive-dwmac 16040000.ethernet: Enabled RFS Flow TC (entries=10)
[   20.257396] starfive-dwmac 16040000.ethernet: TSO feature enabled
[   20.263572] starfive-dwmac 16040000.ethernet: Using 40/40 bits DMA host/device width
[   20.452593] mmc_host mmc0: Bus speed (slot 0) = 45511111Hz (slot req 400000Hz, actual 399220HZ div = 57)
[   20.691861] cdns-csi2rx 19800000.csi: probe with driver cdns-csi2rx failed with error -22
[   20.732033] starfive-dwmac 16040000.ethernet end1: renamed from eth1
[   20.739029] starfive-dwmac 16030000.ethernet end0: renamed from eth0
[   20.892493] mmc_host mmc0: Bus speed (slot 0) = 45511111Hz (slot req 300000Hz, actual 299415HZ div = 76)
[   21.147531] 8021q: 802.1Q VLAN Support v1.8
[   21.340523] mmc_host mmc0: Bus speed (slot 0) = 45511111Hz (slot req 200000Hz, actual 199610HZ div = 114)
[   21.796495] mmc_host mmc0: Bus speed (slot 0) = 45511111Hz (slot req 100000Hz, actual 99805HZ div = 228)
[   22.061485] mmc0: Failed to initialize a non-removable card
[   22.591276] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[   22.628759] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[   22.635477] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'
[   22.886116] starfive-dwmac 16030000.ethernet end0: Register MEM_TYPE_PAGE_POOL RxQ-0
[   22.896562] starfive-dwmac 16030000.ethernet end0: PHY [stmmac-0:00] driver [YT8531 Gigabit Ethernet] (irq=POLL)
[   22.916459] dwmac4: Master AXI performs fixed burst length
[   22.922059] starfive-dwmac 16030000.ethernet end0: No Safety Features support found
[   22.929800] starfive-dwmac 16030000.ethernet end0: IEEE 1588-2008 Advanced Timestamp supported
[   22.938877] starfive-dwmac 16030000.ethernet end0: registered PTP clock
[   22.949986] starfive-dwmac 16030000.ethernet end0: configuring for phy/rgmii-id link mode
[   22.962676] 8021q: adding VLAN 0 to HW filter on device end0
[   22.968538] starfive-dwmac 16030000.ethernet end0: Adding VLAN ID 0 is not supported
[   22.978469] starfive-dwmac 16040000.ethernet end1: Register MEM_TYPE_PAGE_POOL RxQ-0
[   22.988737] starfive-dwmac 16040000.ethernet end1: PHY [stmmac-1:00] driver [YT8531 Gigabit Ethernet] (irq=POLL)
[   23.008483] dwmac4: Master AXI performs fixed burst length
[   23.014059] starfive-dwmac 16040000.ethernet end1: No Safety Features support found
[   23.021813] starfive-dwmac 16040000.ethernet end1: IEEE 1588-2008 Advanced Timestamp supported
[   23.030889] starfive-dwmac 16040000.ethernet end1: registered PTP clock
[   23.040098] starfive-dwmac 16040000.ethernet end1: configuring for phy/rgmii-id link mode
[   23.051634] 8021q: adding VLAN 0 to HW filter on device end1
[   23.057406] starfive-dwmac 16040000.ethernet end1: Adding VLAN ID 0 is not supported

I guess the drivers for PCI support are still missing in kernel 6.9? does USB requires working PCI(e) support?

Will it be included in Kernel 6.10? https://patchwork.kernel.org/project/linux-riscv/cover/[email protected]/

@orangecms
Copy link

and tried the new kernel, still no output in lspci and lsusb :(

I will see if I can get this to work.

I guess the drivers for PCI support are still missing in kernel 6.9? does USB requires working PCI(e) support?

This is twofold:

  • there is a native USB controller from Cadence in the SoC
  • many boards have a USB hub attached to PCIe

So far, the kernel.org upstream kernel does not have the PLDA PCIe controller merged yet.
See https://github.com/starfive-tech/linux/tree/JH7110_VisionFive2_upstream/drivers/pci/controller

@onny
Copy link
Author

onny commented Jun 6, 2024

@orangecms Oh good hint, thank you!

So I added:

CONFIG_PHY_STARFIVE_JH7110_PCIE = yes;
CONFIG_PHY_STARFIVE_JH7110_USB = yes;
CONFIG_DRM_CDNS_DSI = yes;
CONFIG_USB_CDNS_SUPPORT = yes;
CONFIG_USB_CDNS3 = yes;
CONFIG_USB_CDNS3_GADGET = yes;
CONFIG_USB_CDNS3_HOST = yes;
CONFIG_USB_CDNS3_STARFIVE = yes;
CONFIG_USB_CONFIGFS = yes;

but still nothing in lsusb and no reaction to attached USB storage device :(

But some interesting debug stuff

[    0.154186] usbcore: registered new interface driver usbfs
[    0.154277] usbcore: registered new interface driver hub
[    0.154338] usbcore: registered new device driver usb
[    0.582755] usbcore: registered new interface driver uas
[    0.582824] usbcore: registered new interface driver usb-storage
[    0.583029] usbcore: registered new interface driver usbserial_generic
[    0.583112] usbserial: USB Serial support registered for generic
[    0.587943] usbcore: registered new interface driver usbhid
[    0.587964] usbhid: USB HID core driver
[   14.303769] /soc/usb@10100000: Failed to get clk index: 0 ret: -517
[   14.310154] cdns3-starfive soc:usb@10100000: error -ENODEV: Failed to get clocks
[   14.329542] platform 10200000.phy: deferred probe pending: jh7110-usb-phy: Failed to get app 125m clock
[   18.116919] Error: Driver 'jh7110-usb-phy' is already registered, aborting...

@eshattow
Copy link

eshattow commented Jun 6, 2024

and tried the new kernel, still no output in lspci and lsusb :(

I will see if I can get this to work.

I guess the drivers for PCI support are still missing in kernel 6.9? does USB requires working PCI(e) support?

This is twofold:

* there is a native USB controller from Cadence in the SoC

* many boards have a USB hub attached to PCIe

So far, the kernel.org upstream kernel does not have the PLDA PCIe controller merged yet. See https://github.com/starfive-tech/linux/tree/JH7110_VisionFive2_upstream/drivers/pci/controller

Rebase with: PCI: starfive: Add JH7110 PCIe controller https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git/commit/?h=controller/microchip

This is most of the PCIe host series less patch 22/22 riscv: dts: starfive: add PCIe dts configuration for JH7110. Apply and resolve conflicts as the filenames have changed with the reorganization.

Also apply or use as a reference this series for the Kconfig options: v3-riscv-defconfig-Enable-StarFive-JH7110-drivers.patch https://patchwork.kernel.org/project/linux-riscv/list/?series=858940

Not sure about USB yet. I'm reading others' work and did some testing of my own, it does not seem complicated only that StarFive developers had zero motivation to update the documentation and test USB host mode on Linux kernel releases with the new devicetree verbiage as the VisionFive2 product only has this connected on the USB-C port which is used primarily for power and gadget mode. It does not make sense to use that for host mode when they have a VL805 hanging on the PCIe bus which is more capable for USB and is working in upstream.

Edit: Note there's some merge conflicts if you want to follow along. Updated build instructions for 6.10.0-rc3* at that one issue I abuse the Update comment button on frequently

@malteneuss
Copy link
Contributor

Current state of the PR is also booting for me :)

I'd prefer fixing the kernel config once and for all :)

Just to have a better understanding: Is usb the remaining topic that needs fixing for this branch, or is it also planned to backport some upstream patches to get feature parity with the current vendored kernel. E.g. pcie seems to be supported earliest with upstream 6.11

@onny
Copy link
Author

onny commented Jun 8, 2024

usb would be a requirement for me. in case it is possible to backport some patches that would be nice

@onny
Copy link
Author

onny commented Aug 1, 2024

image

yay usb hub gets detected with mainline kernel 6.11-rc1 🥳

no further patches or config flags required. going to test it tomorrow with an external drive

@onny onny force-pushed the starfive2-mainline branch 3 times, most recently from 0d672e0 to 0cf4b17 Compare August 2, 2024 11:59
@onny onny marked this pull request as ready for review August 2, 2024 11:59
@onny
Copy link
Author

onny commented Aug 2, 2024

Updated the PR

Still depends on NixOS/nixpkgs#316724 backported to 24.05. Otherwise we could also merge this with 24.11

@NickCao
Copy link
Member

NickCao commented Aug 2, 2024

Still depends on NixOS/nixpkgs#316724 backported to 24.05. Otherwise we could also merge this with 24.11

Can't we override preferBuiltin for a single kernel?

@misuzu
Copy link

misuzu commented Aug 2, 2024

no further patches or config flags required

Does htop works for you?

% htop
[1]    6894 segmentation fault  htop
% uname -a
Linux nixos 6.11.0-rc1 #1-NixOS SMP Tue Jan  1 00:00:00 UTC 1980 riscv64 GNU/Linux

I also can't build bootstrap-tools on mine vf2

bootstrap-tools> Segmentation fault
error: builder for '/nix/store/avz33cl25cb5gipjfr8yjspargscd8sd-bootstrap-tools.drv' failed with exit code 139;
       last 10 log lines:
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/bin/strip
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/bin/tar
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/bin/xargs
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/cc1
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/cc1plus
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/collect2
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/g++-mapper-server
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/lto-wrapper
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/lib/libpcre2-16.so.0.11.2
       > Segmentation fault
       For full logs, run 'nix log /nix/store/avz33cl25cb5gipjfr8yjspargscd8sd-bootstrap-tools.drv'.

@onny
Copy link
Author

onny commented Aug 2, 2024

@misuzu haven't used it that much yet. would like to test nvme boot at the next step

@misuzu
Copy link

misuzu commented Aug 2, 2024

@misuzu haven't used it that much yet. would like to test nvme boot at the next step

It works (at least with the upstream u-boot)

@eshattow
Copy link

eshattow commented Aug 3, 2024

Linux 6.11rc1 is the first Linux release that includes JH7110 PCIe Host functionality, and the riscv64 defconfig was updated apropos to enable this.

However, I observe some weird behavior in Debian which is what I'm most familiar with.

  1. Docker fails on a vanilla riscv64 defconfig Linux kernel v6.11-rc1. When Linux is built with Debian Kconfig merged to Debian riscv64 Kconfig, actually Docker is then successful. So whatever y'all do for nix it will be probably important as people are going to want Docker and I can confirm that if you have the correct Kconfig options that it is possible to have functional Docker, but this is not the default situation from upstream Linux riscv64 defconfig as tested.

  2. Contents of /proc/mtd are one entity that spans all of the SPI NOR flash when Linux is built with riscv64 defconfig. I don't know what is going on and especially because something is built in when otherwise you would see partitions if it is built as a module... I did not understand fully what the problem is. The expected partition entities are present from devicetree data and visible in /proc/mtd when Linux built with Debian Kconfig merged to Debian riscv64 Kconfig; if you know what the reason is then please share your knowledge, I would like to understand better.

  3. Be more outspoken and do the work of testing all JH7110 platform boards you can get access to if you want USB fully supported; if it "just works" now then this is probably not even what you think it is... and you need to do stuff!

Even if there is "lsusb" output you must take action!

...or else it is likely to get broken by drive-by code contributions from people with the least understanding of the problem.

3a. StarFive VisionFive2. The StarFive developer says "VF2 cadence USB is dr mode device" and in my reading of this the meaning is that nobody is interested in testing USB host mode on the USB-C power input port because there is an actual USB host controller VL805. You there! reading this comment... YES, you! And you have VisionFive2? Do you? You DO!? okay... humor me this time and solve this riddle, I'll bet ya a proverbial turkey leg and a stein of beer that the VisionFive2 USB-C connector can act as USB Host... and you win if you conclusively prove me right or prove me wrong, either way!

3b. Pine64 Star64. This is listed at its product page as "1x native USB 3.0 Host Port / 3x shared USB 2.0 Host Port / PCIe x1 Slot". There's a proposed tuning from U-Boot for this, and it requires testing to verify the USB and PCIe functionality, before extending that testing and possible validation to mainline Linux Kernel.

3c. Milk-V Mars needs testing, from all available product documentation there is apparently one USB port exposed connecting with JH7110 Cadence USB.

3d. Milk-V Mars CM / CM Lite is in progress, I have confirmed that USB and PCIe simultaneous function within U-Boot is possible, and each is possible within Linux (both simultaneous are expected but not yet realized); next steps are to extend this and validate with mainline Linux kernel.

3e. ROMA Laptop (JH7110 edition) needs testing, the one Debian enthusiast I know of that has this hardware does not have any time available to give attention to this.

3f. Forlinx OK7110-C, unknown and did not read of anyone with this hardware yet.

3g. Framework Mainboard JH7110 edition, unknown and did not read of anyone with this hardware yet.

  1. (edit, added) Linux upstream defaults with ZBOOT and Zstd without ZBOOT are problematic for GRUB and U-Boot. The workaround for Debian is --enable CONFIG_KERNEL_GZIP --undefine CONFIG_KERNEL_XZ but that is not prescriptive for nix, it's just something you might encounter as a failure for the kernel to load and it complains about UEFI STUB (which is enabled as it should be).

Are there more JH7110 boards not named? This is relevant because mostly they are derivatives of the VisionFive2 development board design. You get to add support for many boards at a low cost of planning ahead for these minor differences with your support of the VisionFive2 itself.

@misuzu
Copy link

misuzu commented Aug 3, 2024

2. Contents of /proc/mtd are one entity that spans all of the SPI NOR flash when Linux is built with riscv64 defconfig.

It looks fine with pkgs.linuxPackages_testing

% cat /proc/mtd
dev:    size   erasesize  name
mtd0: 000f0000 00001000 "spl"
mtd1: 00010000 00001000 "uboot-env"
mtd2: 00f00000 00001000 "uboot"
% uname -a
Linux nixos 6.11.0-rc1 #1-NixOS SMP Tue Jan  1 00:00:00 UTC 1980 riscv64 GNU/Linux

@onny
Copy link
Author

onny commented Aug 3, 2024

As soon as I put a nvme drive (SSDPEKKF512G8L, Intel SSD 660p Series 512 GB, M.2 80 mm PCIe 3.0 x 4, 3D2, QLC) into it, it stucks at boot with

screenshot_2024-08-03-103145

In case it is relevant, this is my firmware version

  • U-Boot SPL 2024.07 (Jul 01 2024 - 18:07:18 +0000)
  • OpenSBI v1.5

@eshattow
Copy link

eshattow commented Aug 3, 2024

As soon as I put a nvme drive (SSDPEKKF512G8L, Intel SSD 660p Series 512 GB, M.2 80 mm PCIe 3.0 x 4, 3D2, QLC) into it, it stucks at boot with

...

In case it is relevant, this is my firmware version

* U-Boot SPL 2024.07 (Jul 01 2024 - 18:07:18 +0000)

* OpenSBI v1.5

Assuming EFI boot method from U-Boot, as a diagnosis step you may try to interrupt U-Boot, and env set fdtfile to delete the value from the running instance (but not saved across reboot), then boot, from U-Boot. What this (may) do is fail to load the devicetree blob from the named path in $fdtfile and so passing the internal firmware device tree from U-Boot onto Grub or Linux or whatever you have next in the chain of bootloaders. The point being to force U-Boot internal fdt used by Linux and see if that is any different than the dtb you have copied from /usr/lib/linux-image-xyz... into the EFI System Partition. Did you remember to update the dtb's, if applicable?

  1. Contents of /proc/mtd are one entity that spans all of the SPI NOR flash when Linux is built with riscv64 defconfig.

It looks fine with pkgs.linuxPackages_testing

% cat /proc/mtd
dev:    size   erasesize  name
mtd0: 000f0000 00001000 "spl"
mtd1: 00010000 00001000 "uboot-env"
mtd2: 00f00000 00001000 "uboot"
% uname -a
Linux nixos 6.11.0-rc1 #1-NixOS SMP Tue Jan  1 00:00:00 UTC 1980 riscv64 GNU/Linux

Weird for me, the mystery is not solved. Nothing needs done then for you though... that looks fine to me.

@orangecms
Copy link

For the SPI flash to work, you need

  • CONFIG_MTD_CFI=y
  • CONFIG_MTD_CFI_ADV_OPTIONS=y
  • CONFIG_MTD_ROM=y

@NickCao
Copy link
Member

NickCao commented Aug 29, 2024

Anything else blocking this? USB and MTD both seem to work fine with default kernel config. *on nixos-unstable

@Mic92
Copy link
Member

Mic92 commented Sep 1, 2024

Maybe @0x4A6F can re-test?

@0x4A6F
Copy link
Member

0x4A6F commented Sep 10, 2024

Using the resulting sd-image from this PR results in builds breaking (i.e. nom build --system riscv64-linux .#hello):

Unpacking the bootstrap tools...
Patching the bootstrap tools...
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/ar
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/as
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/bash
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/bzip2
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/cmp
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/coreutils
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/cpp
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/diff
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/diff3
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/find
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/g++
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/gawk
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/gcc
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/grep
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/gzip
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/ld
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/make
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/nm
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/objdump
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/patch
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/patchelf
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/ranlib
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/readelf
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/sdiff
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/sed
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/strip
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/tar
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/bin/xargs
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/cc1
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/cc1plus
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/collect2
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/g++-mapper-server
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/lto-wrapper
patching /nix/store/kg61gdq4svvph0c4pcsf867r8nkbiqs7-bootstrap-tools/lib/libpcre2-16.so.0.11.2
Segmentation fault

@onny
Copy link
Author

onny commented Sep 11, 2024

going to give 6.11 a try as soon it is stable the next week

@misuzu
Copy link

misuzu commented Sep 13, 2024

no further patches or config flags required

Does htop works for you?

% htop
[1]    6894 segmentation fault  htop
% uname -a
Linux nixos 6.11.0-rc1 #1-NixOS SMP Tue Jan  1 00:00:00 UTC 1980 riscv64 GNU/Linux

I also can't build bootstrap-tools on mine vf2

bootstrap-tools> Segmentation fault
error: builder for '/nix/store/avz33cl25cb5gipjfr8yjspargscd8sd-bootstrap-tools.drv' failed with exit code 139;
       last 10 log lines:
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/bin/strip
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/bin/tar
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/bin/xargs
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/cc1
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/cc1plus
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/collect2
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/g++-mapper-server
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/libexec/gcc/riscv64-unknown-linux-gnu/13.2.0/lto-wrapper
       > patching /nix/store/lw5iniq79kmqhn3jk40ai7cwqpg659n4-bootstrap-tools/lib/libpcre2-16.so.0.11.2
       > Segmentation fault
       For full logs, run 'nix log /nix/store/avz33cl25cb5gipjfr8yjspargscd8sd-bootstrap-tools.drv'.

6.11-rc7 seems to fix this issue.

@0x4A6F
Copy link
Member

0x4A6F commented Sep 15, 2024

I can also build hello with 6.11-rc7 (with boot.kernelPackages = pkgs.linuxPackages_testing; based on NixOS/nixpkgs/18f300a84b12978aa30432938faff94c08bd550b):

copying path '/nix/store/165dsf8a4zcfvbdqvzc0l0l972f14v33-hello-2.12.1' from 'ssh://[email protected]'
┏━ Dependency Graph:
┃ ┌─ ✔ stdenv-linux-riscv64-linux on [1]
┃ │     ┌─ ✔ bzip2-1.0.8-riscv64-linux on [1] ⏱ 1m39s
┃ │     ├─ ✔ gawk-5.2.2-riscv64-linux on [1] ⏱ 10m56s
┃ │     ├─ ✔ gnumake-4.4.1-riscv64-linux on [1] ⏱ 4m26s
┃ │     ├─ ✔ gnutar-1.35-riscv64-linux on [1] ⏱ 11m33s
┃ │     │     ┌─ ✔ lzip-1.24.1-riscv64-linux on [1] ⏱ 53s
┃ │     │  ┌─ ✔ ed-1.20.2-riscv64-linux on [1] ⏱ 22s
┃ │     ├─ ✔ patch-2.7.6-riscv64-linux on [1] ⏱ 8m36s
┃ │     ├─ ✔ patchelf-0.15.0-riscv64-linux on [1] ⏱ 1m42s
┃ │     │        ┌─ ✔ pcre2-10.44-riscv64-linux on [1] ⏱ 14m26s
┃ │     │     ┌─ ✔ gnugrep-3.11-riscv64-linux on [1] ⏱ 5m45s
┃ │     │  ┌─ ✔ binutils-wrapper-2.42-riscv64-linux on [1] ⏱ 2s
┃ │     ├─ ✔ gcc-wrapper-13.3.0-riscv64-linux on [1] ⏱ 3s
┃ │     ├─ ✔ diffutils-3.10-riscv64-linux on [1] ⏱ 7m46s
┃ │     │     ┌─ ✔ xz-5.6.2-riscv64-linux on [1] ⏱ 3m31s
┃ │     │  ┌─ ✔ coreutils-9.5-riscv64-linux on [1] ⏱ 27m10s
┃ │     ├─ ✔ findutils-4.10.0-riscv64-linux on [1] ⏱ 12m49s
┃ │  ┌─ ✔ stdenv-linux-riscv64-linux on [1]
┃ ├─ ✔ version-check-hook-riscv64-linux on [1] ⏱ 1s
┃ ✔ hello-2.12.1-riscv64-linux on [1] ⏱ 2m40s
┣━━━ Builds            │ Downloads         │ Uploads   │ Host
┃        │       │     │     │       │     │     │     │ localhost
┃        │ ✔ 138 │     │     │ ↓ 310 │     │     │ ↑ 2 │ [1]: ssh://[email protected]
┗━ ∑ ⏵ 0 │ ✔ 138 │ ⏸ 0 │ ↓ 0 │ ↓ 310 │ ⏸ 0 │ ↑ 0 │ ↑ 2 │ Finished at 10:40:24 after 15h42m47s

The kernel logs spit out these kinds of messages:

[42347.981340] swiotlb_tbl_map_single: 6 callbacks suppressed
[42347.981362] dwmmc_starfive 16020000.mmc: swiotlb buffer is full (sz: 360448 bytes), total 32768 (slots), used 80 (slots)
[42347.999037] dwmmc_starfive 16020000.mmc: swiotlb buffer is full (sz: 360448 bytes), total 32768 (slots), used 264 (slots)

@eshattow
Copy link

The kernel logs spit out these kinds of messages:

[42347.981340] swiotlb_tbl_map_single: 6 callbacks suppressed
[42347.981362] dwmmc_starfive 16020000.mmc: swiotlb buffer is full (sz: 360448 bytes), total 32768 (slots), used 80 (slots)
[42347.999037] dwmmc_starfive 16020000.mmc: swiotlb buffer is full (sz: 360448 bytes), total 32768 (slots), used 264 (slots)

In my experience this MMC full buffer error happens with Linux Kernel that is running with devicetree data of internal U-Boot firmware device tree instead of the devicetree for that Linux Kernel.

Troubleshooting MMC buffer errors in kernel log for JH7110 board:

  • Is your installed version of U-Boot v2024.10-rc1 or more recent? Important for the U-Boot EFI support which is now more consistent in loading devicetree data from named path or path built from EEPROM-based board detection heuristic.

  • Have you installed devicetree files for that Linux Kernel to the EFI System Partition so that U-Boot can load them? These must be in directory path "dtb" within root of EFI System Partition.

  • Is U-Boot correctly loading the named devicetree file for your board?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants