Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Pwnagotchi image buildable #1120

Open
wants to merge 15 commits into
base: master
Choose a base branch
from

Commits on Oct 3, 2022

  1. Robustify Makefile

    Fixes:
    - Updates packer version
    - Updates packer-plugin-arm-image name
    - Downloads packer plugin via packer CLI instead of building from source
    - Supports image builds on more machines by downloading the correct
      packer release for the current build machine's CPU type
    - Dependencies added to 'image' target to ensure that packer is installed
      and that the image is only rebuilt when the input files are updated
    - Prevents Ansible from changing build machine's hostname
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    aed23b2 View commit details
    Browse the repository at this point in the history
  2. Update apt-get command to support oldstable

    Raspbian has been deprecated in favor of RaspiOS.
    As a consequence, the Raspbian repo has changed from 'stable' to 'oldstable'
    which causes the apt-get update command to fail.  This is the minimal change
    required to get the outdated image to build correctly.
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    a7f03e3 View commit details
    Browse the repository at this point in the history
  3. Update re4son-kernel repo key

    The PGP signature file for the re4son-kernel repo expired on August 29, 2022.
    The re-signed PGP key is available from the MIT PGP keyserver.
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    e220cce View commit details
    Browse the repository at this point in the history
  4. Update requirements to fix broken imports

    flask, Jinja2, Werkzeug, and itsdangerous are all from Pallets Projects
    who seemingly refuse to use PEP-440's ~= "compatible release" clause.
    Instead, all of their project dependencies are ">=" which eventually
    causes things to break.
    Their canned response is to recommend always staying on the latest version
    and to use something like pip-tools to pin dependencies.
    pallets/markupsafe#282 (comment)
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    fb07032 View commit details
    Browse the repository at this point in the history
  5. Bump RasPiOS to latest buster release

    Raspbian has been renamed to Raspberry Pi OS.
    RasPiOS on buster is considered legacy but images are still released for it.
    A few packages have to be uninstalled (from the main repo) and
    reinstalled (from the re4son-kernel repo) to fix dependency issues.
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    13769f6 View commit details
    Browse the repository at this point in the history
  6. Update qemu_args to use correct CPU

    The default ARM CPU emulated by qemu is armv7l.  While this is fine for most packages,
    it leads to the wrong opencv and tensorflow being installed.  By emulating the correct
    CPU, we're guaranteed to end up with the correct binaries.
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    e4ad0b9 View commit details
    Browse the repository at this point in the history
  7. Fix broken monitor interface

    If the monitor interface is created before the wlan0 interface is up,
    the resulting mon0 interface will be in a permanently broken state.
    Since the "[email protected]" is disabled and getting timing guarantees
    from systemd is a challenge, we manually up the interface.
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    889ddab View commit details
    Browse the repository at this point in the history
  8. Fix broken wlan0 interface config

    An interface in static mode is required to have an address defined,
    otherwise ifup will throw an error and fail to bring the interface up.
    The Debian wiki provided the pre-up/post-down solution:
    https://wiki.debian.org/NetworkConfiguration#Bringing_up_an_interface_without_an_IP_address
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    ea827a4 View commit details
    Browse the repository at this point in the history
  9. Move system file install into setup.py

    The installation of system files has been moved into the 'install' command
    instead of running unconditionally.  This makes it possible to create a source code
    distribution of that doesn't immediately clobber the build machine's system files.
    This will also allow the image build to be simplified as it will only need to copy the
    sdist archive instead of multiple directories worth of files.
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 3, 2022
    1 Configuration menu
    Copy the full SHA
    3af03cb View commit details
    Browse the repository at this point in the history
  10. Copy requirements.txt to requirements.in

    The idea here is that the requirements.in file should be the loosest allowed
    requirements for the project itself, then it should be pip-compile'd into
    a concrete requirements.txt that has known working values.  This prevents
    unnecessarily pinning packages to specific releases which makes it easier
    to keep package versions up-to-date going forward.
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    fd136eb View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2022

  1. Document and update requirements.in

    Firstly, all requirements were sorted.  Then, any dependencies not directly
    used by pwnagotchi were removed (e.g. gast isn't used directly, but it will
    eventually be installed as an indirect dependency of tensorflow).
    Next, every single dependency was researched and documented to determine how
    it's used in addition to what versions it can be safely upgraded to.
    Most dependencies have been updated to use the PEP-440 "compatible release" feature.
    Lastly, the --extra-index and --prefer-binary options have been added.
    As a result of some unfortunate publication issues (e.g. grpcio v1.46.X),
    some non-yanked libraries simply cannot be built from source.
    The --prefer-binary option allows pip to prefer an older library version if
    it has a binary wheel available.  This also results in faster builds as
    fewer requirements actually need to be compiled from source.
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 8, 2022
    Configuration menu
    Copy the full SHA
    a65d89f View commit details
    Browse the repository at this point in the history
  2. Add pip-compile'd requirements.txt

    In theory, this file shouldn't need to be regenerated very often.
    One thing of note is that pip-compile strongly recommends that it be run
    on the target system in order to ensure it selects the correct versions.
    This is because pip-compile doesn't (yet) have a way to specify the
    Python version, platform, and machine type during compilation.
    The setup.py file was also tweaked to ignore any --options in the
    requirements.txt file.
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 8, 2022
    Configuration menu
    Copy the full SHA
    339a219 View commit details
    Browse the repository at this point in the history
  3. Add libgtk-3-0 for opencv

    Despite the fact that a headless version of opencv exists,
    stable-baselines and gym both request the regular opencv which
    has a dependency on libgtk3 despite the fact that they don't use
    any of the GUI functionality.
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 8, 2022
    Configuration menu
    Copy the full SHA
    2d2ccd8 View commit details
    Browse the repository at this point in the history
  4. Install pwnagotchi from local sdist

    Installing pwnagotchi using the local sdist has a few key benefits:
    - We're no longer installing all of the data files twice,
      once in packer and once in the setup.py script.
    - The image is built using the code in the current local repo,
      not whatever code is currently pushed to the remote master branch.
      This makes it much easier to create and test custom images.
    - Installs pwnagotchi using pip, just like the auto-update plugin does.
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 8, 2022
    Configuration menu
    Copy the full SHA
    c63cbe0 View commit details
    Browse the repository at this point in the history
  5. Actually clean the apt cache

    As the added comments describe, the options available to us from Ansible's
    apt module don't actually clean the apt cache.
    Manually running apt-get clean recovers around 400MB of space.
    
    Signed-off-by: llamasoft <[email protected]>
    llamasoft committed Oct 8, 2022
    Configuration menu
    Copy the full SHA
    58095c8 View commit details
    Browse the repository at this point in the history