Switch from subprocessing to gpg2 to PyMe

[micahflee]

GPGME is GnuPG's official API, and much more reliable than subprocessing gpg2 and trying to parse the output. We should use it instead. PyMe is the python module that interfaces with GPGME. Both are official GnuPG projects.

We should switch to those to be much more future-proof. Implementing this will also make #75 no longer a problem.

[micahflee]

I just ran into a new error:

This is in Debian 9 (technically in Qubes, in a Debian 9 AppVM). I made it so the GnuPG class was initialized in debug mode. Here's the output:

Syncing: First Look Authority 0x9BB29FF9FD3ED09F
Forcing sync
gpg args ['/usr/bin/gpg2', '--batch', '--no-tty', '--homedir', '/tmp/tmp41178yzq', '--recv-keys', b'86EB84C96B2E62676B47C4919BB29FF9FD3ED09F']
Fetching public key 0x9BB29FF9FD3ED09F First Look Authority
stdout b''
stderr b'gpg: keyserver option \'ca-cert-file\' is obsolete; please use \'hkp-cacert\' in dirmngr.conf\ngpg: key 9BB29FF9FD3ED09F: "First Look Authority" not changed\ngpg: Total number processed: 1\ngpg:              unchanged: 1\n'
gpg args ['/usr/bin/gpg2', '--batch', '--no-tty', '--homedir', '/tmp/tmp41178yzq', '--armor', '--export', b'86EB84C96B2E62676B47C4919BB29FF9FD3ED09F']
stdout b'-----BEGIN PGP PUBLIC KEY BLOCK-----\n\nmQINBFbD0+cBEADD8tk+LJLti+pEAPOf4kGFs/Wyb2qE2KkwyCfJZMGYdC3tH0hK\nb5INzTUtl+7Pn/TsWGaK881NrO97O1BWM0bqIiOVSlwVYV1DOoUeIzb+U1Xv6D7H\nQblk0sSy4jfQ+zydQVIur1oYcsvx1NNbhck/y0+DAQn9BmCTLiTU3xHdlZ4wPUQ8\n/u9Iloph0JCP3gIbW5lIt5jkNYnlBRZ5Amnxnci4G5xKSPcxaTWBcnkPdYez4i9H\n8Teh8sRsCvlZn+2p5i9Z9JCYEDMzFvOH7eyRfiqXaKAXd+3nRgjH5X7fYTmvspAU\n83cXtR/5lR8I7vNLfTYTBuocsNyP06m6YtVPuaKxqQZhfgSA7/mWtXPBBArbg9AJ\nf/RZr8Cw57cJcfNb22myOIZw7f6vWdZcM4/AgYhBct3HxM2jTRIZhIXiPPUpSiLr\nMupVbjTPedh3BKilzoTN7ypKdv3iJ8SVySDYXDa9u0Hx4rlFpo9fPTkk6HIElyjY\nc3unI/j0iRwoL+IZ24nTZUAhjcX1ob5VpnI/YeMLmaIMX1drIpkaVuKw00iBjDhn\n+Zz9lMWWzM8CiWuxooq9ZHGKAq1u9XxRgv5VGDUUXjieXMQ8coOXqbzYbzLjdVkn\ncmfQ/65i8HiAY+A/qq20D0605oRDbUc0H+lETjig654nuLwHUyzsiiZXlQARAQAB\ntBRGaXJzdCBMb29rIEF1dGhvcml0eYkCGwQQAQoABgUCVuCfMQAKCRCDg8tEqTY2\nkiUvD/dAWG0ldCw+PgOqo1mTxI4Eg2SqxPMPVlOeqwV0FY9ydgVuY4Ni8b04nS30\n8j7ni4plRZ06g5DPfBjiNbScqtu5K2b52EdIfVy8a+FfxEZtLJs9DFqTImHaDVaD\ntGQ3iZOmmld45LBxt2aPgJ8qeykymKfkKI6TVhrbPdEkY5qxvjip7TrfLJhimNH+\n4/AckR8HZ9BU3x31TBirfMd7hnZc1ZoesrGZRKXfqaHL7grI+/hUzW5u3/wfD1H5\n8fdLTS0DiE+kXGMKnR3G7WUa9RIodMt/5ruYMcEzLXiZdKC/NXGPCR60qRU37g10\nT6K2+i6k9ZTc4/xCcJx4J7XNgbtPNouJ+ZyjkQ5cmWfoNdBpUvmqRB5dYftaTR19\n7UGpKmn/jyHbI2KhRBsW2Jm69AkwJK5FhnZUt1C5DAwSkf8VBf31TKoQCChnknLu\nfGJJyjjh3ERVgieXZKFL2OhYrQFxnH1X6rFLtrmxyBzqeB0uZTsGlLrb23w77Jz/\nCq27WkQWrrit2R96BKYF7UC5hghdbga4eZslTnqaPChNcuh8NFTl8dZ0lLOqn4VE\n8JgC4CmgjSM06elqYZbhcoSMEANYUkB1saoKVqPqofQZdRISQjqd0tM/621hzNzK\nezoNIe/7oXazyOpD/lzJL1B6vgOZuuppqkPVq3vi7ZgrFkFPiQIcBBMBCAAGBQJW\nxRdBAAoJEEA8JlfNmU9z/FsQALDjOuwcQ4J6cag1bWLfBn6aZvin/nszmJCDWAPc\n3+3Mlr4+W1SAe6G79zaRLk5eEpVx4aQstH1UwcFuK9K9dF+xDcSG40E735gMPGvA\n2Cx5QolytccVklX0J3dRDVYiuMJpRU8PKuxX9RDfXML/w4H0L31fbNZuQ0i7Cam3\nexTZfMsb/vL0yfmoMz93BvHs1BCSPzlczbwXVpIAr1krYkSBkpahOpE7qorLbWvO\nPdDCtGVM0AbJ/jb9w4HNTBA//1PCcVsq22YcQ/H9NlYL1znunM5picDj6XQUrY0c\nO2Qb1olCa/b94h6i2yShiHbJpRH+EeavJKPr7/LNRrZgsYa6NE2USTtIt+SOV7ij\njXUY4H4Jb2bh1S/nsVbDJ7S8yEUZliIBmESN6rQfCDeeLOZPGseeg7FhItxOMGP6\nO3VYQKZ9aUASDrSc4vSM/SHePwcTPhNOLttK9Nm4koUx/i3a9nFjRTJoe/T3TH/U\n8INv2Z3bSTntr6KXKPOJcMGhw0diNX9xCCEvOtrJ9MeOU9TKvGthrCc7G6RfbDXx\nkBINQfyfac2j70YIsHmaNcjacpzQLdVOYL4OLlyw8Zs5qmJcotaKHVCNLWfVZTRj\n330ZF2WUAGHjoV0zXNKXL6Z3Ae2e2iTCZB5PbOVrUeKUqJqR2t8EangBSFvtiKmU\nVVdyiQIcBBMBCgAGBQJX+8sbAAoJEEpuaVITpSTjSlkP/j7xHf2wKvZPjx0qUORP\nTeAXm2vtIvwm0g0sr5LjuOQq1qyyawRICxyJ6IIerdSGCpcbhroBxwPr9DHL+uyK\nKk5Ewd6Mi0yR6SwjAgfGoRC4HYu+QUUzL35ntpVmAEzlfEnDmCVapM4qst+1IeHs\nU4obI4LI+95b1alLyTal32yZqQlXr7TO6D8tslaG5+XgglcKo5wJ9cID7qTJFxVD\nU+tJKrn93Xim3/oj8pbux9BUecSIToWyn9GazagKZSQIHHA8Ei5yplU07/Dw/73v\nEkPzmvDn2iy1xwlqVa8HkmteuAyi5YQBuk7rQyIenmn7apxNzLiuWlHQDJ1481M4\nbFz4Ara861Zw5NJYHmgv9gvpQ8fByRlvmGOuo4sG+i77AqGJxNl8SA1DmVVFf3/6\n9DGcV6vaB60Gb2dYTIIeRQfx+88x6v93cn7sgj4BlaTT23fxVMXutwfTqBJvDfrp\n+M3A6ZTA9FMH1P4SKBlMP1tAGw6s/ykrO+WUCmOEKscAbvO+ix/+9l0Gb3MxQCoA\nXa9Uytev91nMAMgOVDyJHZEIGCs5CSMhu+5JbrylfCpzBuPNzGe4BU8zjgTYGn6N\nK91nLLBZXbzc0WEnTTqnVIH9bVMRuWUqAHE/CseZnVE6cE1LiQjkB3nc0yAK5Xa2\nyiS7KdlvGGMq30TapT3b2ecHiQI9BBMBCgAnBQJWw9PnAhsDBQkB4TOABQsJCAcD\nBRUKCQgLBRYCAwEAAh4BAheAAAoJEJuyn/n9PtCfVnsP/Am1ltfTx+OPAYh+0788\n5SIimfIpVUR0nviRrNtR5bfynnrW5bcPuPw+u0OTe5iPmmQ0q668vF/UlyWged6E\nL3qcc5qwjAOyC+LO8oo12IrXhVXWNUSpM5cSrUhPNphzl1a7Uim77kgKbE8MNY78\nBjUtej8LOms8y9sUiQ3dJC/uJKD6Z5VNdK5ITaN+4Vbk8tnjL07LGERsi9Mwz5eJ\nG9ra209y3BFiSxVuwMpzUuGfYm5pki0kVXiDLMuu1aDOLmwg/hwJKxq5OkaYNSqW\neISMnKEHKFZz1tm8r8Sv85kYsy3keyfWrl7I6dsSw/RSBNk/EME9xPLTT+E86wi1\nHsu1vpBUofzobY8NNaTfUE1ui6jofB4qh4lmSzFe0u01n4j3VRsgZPJcpBa9SqlF\nnilMjGcrVwSe6iUVJATKkrRm73k5OcKyuxWz015FPzKeH7+Pna9VZy58FZzk7ZBT\nqXkQp9qseDPbuDB9ZOJP9MKYhglj7XCPTTSzRYm8716b+GGAB0MwQ7arILhbGZpa\njQXqmc0aRYgGzRaJG4T7P1rLhU2uLV8+EehQ5H72Tm5yQ+Oj7jfjNOT2aRtP7pTQ\nzVx6MtkKuQFxx2Wds28vFjlV/s2VmMqZPWz/SG54GkrgDHqlZTJvAqTWPqvjjF9Y\nvTL0xSzW+ad93TT7EASKN2VquQINBFbD0+cBEAC0/ZTDfjZ54WYd1HGMb/k2TRQD\n2ouH0wmXxVTzYsFQ59qKg+Dsiu29olpyiOMFUtmNZZ6zft9ifoFc5QRj00BI1fts\ny2+Jp4fFBrNOuGtPc+U1KTybDYG6pO/ki0mpRg5GMialsmmzlWkG+IL3r0hKC4Hb\nFWXHEDWoyb27AjaH8R5nOu6KT2aQwl02jd6KNQ6MA/OrPnQhXi7wzLwRrK889d05\nEXKNry+SaAAP3KWynM8yf1NNpYBapJcPYjdjhmT5Gr9SEYyO0e5ybjeg5czc3qjQ\nQv/c6XGqBhWwt9clPU4eGeXK5FDD5G0HU0IUcT62npobyikL6i9tbrw7pXxRNFDk\n5JyoLqT58yg1gq3tac1jo0n4aj/+t1z2B6S1O6P/vvQsRnI8ugpg2OScsAcCfstr\nWJq80+1fSjf3ODqzLi5VyS7N+3hCtAO6Ytpou/oJ9C5dH3n2Z8ZqBKm9UbOv0ekI\nNe8/QpyxCQYfxtGVB/39pGwiL/dVchepBRH6F+LVqqflFOI7H9JvaAiF1hp4/hP1\nk5jQVXM4m7Q3rgqNE40Xnha5qHlT0NaRgdIRz3sOFkJ+MgQaKX/uPkhnUyXZ9hTm\nxSu2wZ5l7lHKiSjqAX+MWpE421VPwva6veFI/4UDuQpOE5jvZc/zCWMhjh5sfrgv\n0uoME4MD7IOLJkfICQARAQABiQIlBBgBCgAPBQJWw9PnAhsMBQkB4TOAAAoJEJuy\nn/n9PtCf3KYP/Rss1lsn132PiZkZ4ABbzc/p4RTAcr7Dcf+PoOYYTdmIjGeLNxKl\nkcE0Wy2l8nCwXfG6amqcAhCWW2F6JKF9gxlZNiaqlw5g0RKB14DSJCuD6S+QOH6v\nL/+geV4J4/+ryQbPW1ZD0WQZKIguCVr5eFrqgtaw25nYPn46s7IWkIH1zswMzh5e\nUD/lOQg6KLbmx6wzgTVffzBNbRutdGwIYve53KdkiWd2hKBoQEzPusrTWLrGm6bu\nnehoVPZJ8KebNLrMzMWvhkmxTGBxX0ifo9pnk0/AEHf9Ubrk3PD38h0E5OPd9qgN\nnKoFi8RNVknO7PFQKJL2laJqHWsA9P2Gbs2d9QF3MaEitLTj+Y3mNOfgLxyrYt3N\nMYgUwAFpT3GLq7esLj5H6Qr4YYEplUbS6D048g7NZC8125P6XhYaQtDGFD38BhkV\nEYDhDZa1SZF0jPUSfYMW8sU7HPDWZmyonuhuy3KklF2jjPHiCrte40apTMgeRR9T\nRGJxsGQI/Z455CIk+e9+prBOYCdfeAyvPiXg4W29umI/H6ULfCli7RYW93m/3uZQ\ntkFyP/iqgHay120h+YGFvsk5sABjNfV2cKS/68zZ2+dzDhYEcKcBXZZdeCHVtplf\nCBzgHfOwsAFIUqOgbr03rlmEEFNv8cU5zvZ/Z2WJJNByXjHVE7I0UYn7\n=TaIJ\n-----END PGP PUBLIC KEY BLOCK-----\n'
stderr b"gpg: keyserver option 'ca-cert-file' is obsolete; please use 'hkp-cacert' in dirmngr.conf\n"
importing keys into default homedir
stdout b''
stderr b'gpg: key 9BB29FF9FD3ED09F: "First Look Authority" not changed\ngpg: Total number processed: 1\ngpg:              unchanged: 1\n'
gpg args ['/usr/bin/gpg2', '--batch', '--no-tty', '--homedir', '/tmp/tmp41178yzq', '--with-colons', '--list-keys', b'86EB84C96B2E62676B47C4919BB29FF9FD3ED09F']
stdout b'tru::1:1480186931:0:3:1:5\npub:-:4096:1:9BB29FF9FD3ED09F:1455674343:1487210343::-:::scESC:::::::\nfpr:::::::::86EB84C96B2E62676B47C4919BB29FF9FD3ED09F:\nuid:-::::1455674343::B446F07E9ED5DB8A9B86B8B02FC0C4E49A84098E::First Look Authority:::::::::\nsub:-:4096:1:55BF11904F632ADB:1455674343:1487210343:::::e::::::\nfpr:::::::::12EC3896457B69FBD2DB536655BF11904F632ADB:\n'
stderr b"gpg: keyserver option 'ca-cert-file' is obsolete; please use 'hkp-cacert' in dirmngr.conf\n"
Downloading URL https://raw.githubusercontent.com/firstlookmedia/gpgsync-firstlook-fingerprints/master/fingerprints.txt
Downloading URL https://raw.githubusercontent.com/firstlookmedia/gpgsync-firstlook-fingerprints/master/fingerprints.txt.sig
gpg args ['/usr/bin/gpg2', '--batch', '--no-tty', '--homedir', '/tmp/tmp41178yzq', '--keyid-format', '0xlong', '--verify', '/tmp/tmpze2_9xww', '/tmp/tmpyabyneoa']
stdout b''
stderr b'gpg: keyserver option \'ca-cert-file\' is obsolete; please use \'hkp-cacert\' in dirmngr.conf\ngpg: Signature made Wed 23 Nov 2016 02:50:18 PM PST\ngpg:                using RSA key 86EB84C96B2E62676B47C4919BB29FF9FD3ED09F\ngpg: Good signature from "First Look Authority" [unknown]\ngpg: WARNING: This key is not certified with a trusted signature!\ngpg:          There is no indication that the signature belongs to the owner.\nPrimary key fingerprint: 86EB 84C9 6B2E 6267 6B47  C491 9BB2 9FF9 FD3E D09F\n'
gpg args ['/usr/bin/gpg2', '--batch', '--no-tty', '--homedir', '/tmp/tmp41178yzq', '--keyid-format', '0xlong', '--list-keys']
stdout b'/tmp/tmp41178yzq/pubring.kbx\n----------------------------\npub   rsa4096/0x9BB29FF9FD3ED09F 2016-02-17 [SC] [expires: 2017-02-16]\n      86EB84C96B2E62676B47C4919BB29FF9FD3ED09F\nuid                   [ unknown] First Look Authority\nsub   rsa4096/0x55BF11904F632ADB 2016-02-17 [E] [expires: 2017-02-16]\n\n'
stderr b"gpg: keyserver option 'ca-cert-file' is obsolete; please use 'hkp-cacert' in dirmngr.conf\n"
Verifying signature
Syncing complete.

The signature verified successfully, and it was in fact signed with the right signing key. I think this is just an error parsing output, in my particular setup with my particular gpg2.

So I think that switching from subprocessing out to gpg2 to using PyMe will solve this problem too. @samuelcouch wanna work on this issue?

[samuelcouch]

Yep – I'm working on the JSON settings rebase first, and then was going to integrate PyMe after that (or somewhat in parallel). I'll check it out then.

[samuelcouch]

https://www.gnupg.org/blog/20160921-python-bindings-for-gpgme.html this references a pyme3 that supports Python 3, however pip says

Could not find a version that satisfies the requirement pyme3 (from versions: )
No matching distribution found for pyme3

Going to hold off on this until there's better support for python 3

[micahflee]

If we're delaying on this, I'll remove it from the 0.1.1 milestone.

[ageis]

There are major advantages to subprocessing IMO because many of the modules don't provide access to all of the routines. Including GPGME.