Finding sources of European funding repository (https://ec.europa.eu/)
Note: Initially it will take a few seconds until the website is loaded as it is loading the European funding database for the first time.
We are going to develop a simple Python script to extract the database of European calls for funding and perform the search on the titles or tags of the different open calls. To do this we connect to the database in .json format, convert the fields that interest us into a data frame, select the open calls and perform searches from keywords.
Once the search script is built into the dataframe extracted from the .json file, we use the Streamlit library to develop our web application from the script.
With Streamlit (st.markdown(), st.image(), st.title(), etc)we can mark elements that we want to be part of the main website and we can put a sidebar, where in our case we will add the fields to enter the keywords and the elements to determine where to look for the dataframe (st.sidebar…). Later we will determine with a multiple condition “if-elif”, which will show us the web based on the marked elements.
We must save the file as app.py (it can be downloaded in this format from the Jupyter notebook itself).
Once the script is developed, we deploy it on a website with the Streamlit library and test it localhost.
Deploy your Python script web app (Streamlit) on Heroku
After testing that our European-funded search Python script works on localhost, we’re going to try to deploy it to the web using Heroku.
Heroku is a cloud platform where you can build, operate and run web applications with different languages, also offers free plans to test the webapps developed. Another advantage is that you can publish your GitHub repositories directly to Heroku which helps to keep your code and collaboration processes up to date.
For this we must include in our repository, next to the app.py where our script is executed, 3 more files:
- Procfile (acquisitions file): Procfile contains the code that provides the commands to indicate which files the application should run when it is opened.
web: sh setup.sh && streamlit run app.py
- requirements.txt (requirements file): contains the list of packages and dependencies required to run the web application
numpy requests pandas streamlit
- setup.sh (installation file): contains a shell script needed to configure the shell environment for our purpose
mkdir -p ~/.streamlit/ echo "\ [server]\n\ headless = true\n\ port = $PORT\n\ enableCORS = false\n\ \n\ " > ~/.streamlit/config.toml
Once we have these files in our GitHub repository, we go to our Heroku user dashboard (previously we will have to register for free) and select “Create new app”.
Give the app a name, using a medium hyphen to separate the words, select your geographic area and you can create the app.
Once created, it can be linked to your GitHub repository, giving the corresponding permissions.
Once we link with our GitHub user, we search for the repository and connect.
It is advisable to select “Enable Automatic Deploys” so that the changes that we implement in the GitHub repository are automatically moved to the webapp that we have just created.
Once this is done, the “Build Progress” starts in the “Activity” section and you will have to wait a few minutes for the repository to load and the webapp to be deployed. It may happen that the Build process fails for some aspect that we will have to solve by looking at the logs. For example, it is easy for us to have to indicate the specific version that we have used in the Python libraries.
When everything is correct, it will indicate “Build succeeded” and you can open the application to check it.
You can view the full repository on Github