I’d like to automate (as much as possible) the build of a CTFd instance in AWS and the setup/config from an import.
Cloud infra is stood up with Terraform, and includes syntax to get CTFd installed and running via container.
This leaves me at a few manual steps where it seems UI interaction is needed to create the initial admin user (few UI prompts) and UI navigation to import the configuration (JSON files).
I’m hoping there is a way to automate this ^^ portion. I didn’t see (on quick glance) anything in API that would drive this and it seems this must be done before importing config (which I need to determine if I can do via API or other automation).
Within the container I’ve found import.py which looks promising. I’ve taken my CTF export, transferred it via S3 to the host and after docker exec’ing into a shell, run python import.py ctf_export.zip. This errors, however (below).
At the time of the import, I’ve not created admin user (or any settings at all) in the UI or taken other steps (goal is for zero interaction). I’ve read elsewhere that if admin users are created in the UI then importing the zip is problematic as it contains admin user (and other duplicative) info also.
Anyone worked though this? Possible?
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
* Loaded module, <module 'CTFd.plugins.challenges' from '/opt/CTFd/CTFd/plugins/challenge
s/__init__.py'>
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
* Loaded module, <module 'CTFd.plugins.dynamic_challenges' from '/opt/CTFd/CTFd/plugins/d
ynamic_challenges/__init__.py'>
* Loaded module, <module 'CTFd.plugins.flags' from '/opt/CTFd/CTFd/plugins/flags/__init__
.py'>
This file will be deleted in CTFd v4.0. Switch to using `python manage.py import_ctf`
zipfile.BadZipfile: zipfile is invalid
Traceback (most recent call last):
File "/opt/CTFd/import.py", line 14, in <module>
import_ctf(sys.argv[1])
File "/opt/CTFd/CTFd/utils/exports/__init__.py", line 129, in import_ctf
raise zipfile.BadZipfile
zipfile.BadZipFile```
Hi Larry,
Some people use the importer to create copies of a CTFd instance. You are correct that the import will contain the details of users however the passwords and such will be hashed.
One thing you could try is using CTFd’s scripting interface with app and app.app_context() to create scripts that will do automation for you.
Another thing you could take a look at is GitHub - CTFd/ctfcli: ctfcli is a tool to manage Capture The Flag events and challenges which lets you create challenges from a YAML specification.
As for your zip file I can’t tell why it would be invalid since I haven’t received any other reports of invalid zip files lately. I’d need to understand more about this specific situation.
I’ve tried also importing my config export using python manage.py import_ctf ./ctf_export.zip and again receive errors for the source file (same as the previous mentioned command).
If I perform the CTFd install, add a admin user via the UI, and them import the same zip file from the UI, it works successfully and data can be viewed afterward.
root@2f8eb7f499f8:/opt/CTFd# python manage.py import_ctf ./ctf_export.zip
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
* Loaded module, <module 'CTFd.plugins.challenges' from '/opt/CTFd/CTFd/plugins/challenge
s/__init__.py'>
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
* Loaded module, <module 'CTFd.plugins.dynamic_challenges' from '/opt/CTFd/CTFd/plugins/d
ynamic_challenges/__init__.py'>
* Loaded module, <module 'CTFd.plugins.flags' from '/opt/CTFd/CTFd/plugins/flags/__init__
.py'>
zipfile.BadZipfile: zipfile is invalid
Import Failure:
1674677582
root@2f8eb7f499f8:/opt/CTFd# ```
What can I provide (or zoom?) to help resolve, assuming this may be a defect...
There are only a few reasons why CTFd would report BadZipFile. You can see the two cases here: CTFd/__init__.py at master · CTFd/CTFd · GitHub
You’d need to investigate to figure more about why CTFd is rejecting your zip.
It was recently reported that the malicious zip file detection was a little overzealous and needed to be fixed but I haven’t gotten around to it. Some imports that have files with .. in the name are being rejected when they shouldn’t be. I haven’t gotten to fixing this but I feel like this is a rare occurence. It might be what you’re running into though.
Hmm. I looked at that code earlier and if either of those factors were in play I’d expect the error to include ‘zipfile is malicious’ or ‘zipfile is too large’. As I’m only seeing ‘zipfile is invalid’. And given the same zipfile successfully imports via the UI, I conclude nothing is wrong with the zipfile. Am I overlooking something?
Are the zips that you are using correct? The only reason that I can imagine that the import works via the UI but doesn’t work via the CLI is if the zips are different or malformed.
This is all sorted and was in fact a user error and was related to the directory on the host (that the zip copied to) not being accessible by the container. The error indicated invalid zip, when in fact, I believe, the container couldn’t access/read the zip at all. So, perhaps the error could differentiate between invalid (inferring accessible but something wrong with it) and unable to access?
In any case, all is working!