Month: January 2021

Gevent is the use of simple, sequential programming in python to achieve scalability provided by asynchronous IO and lightweight multi-threading (as opposed to the callback-style of programming using Twisted’s Deferred).
It is built on top of libevent/libev (for asynchronous I/O) and greenlets (lightweight cooperative multi-threading).
we can use pip install gevent to install it.
Here is the sample.py:

from gevent import monkey
monkey.patch_all()
from flask import Flask
from gevent import pywsgi
app = Flask(__name__)
@app.route('/')
def hello():
    return "Hello World!"
if __name__ == '__main__':
    #app.run()
    server = pywsgi.WSGIServer(("0.0.0.0", 8080), app)
    server.serve_forever()                            

you can run it in background:
nohup python3 sample.py &
then you can open the sample app on http://your_server_ip:8080

Use ApacheBench To Do Load Testing:
1. apt-get install apache2-utils
2. ab -r -n 2000 -c 200 http://127.0.0.1:8080/?delay=1

Concurrency Level:      200
Time taken for tests:   2.470 seconds
Complete requests:      2000
Failed requests:        0
Total transferred:      294000 bytes
HTML transferred:       24000 bytes
Requests per second:    809.60 [#/sec] (mean)
Time per request:       247.035 [ms] (mean)
Time per request:       1.235 [ms] (mean, across all concurrent requests)
Transfer rate:          116.22 [Kbytes/sec] received

compare it with sample_old.py without gevent:

from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello():
    return "Hello World!"
if __name__ == '__main__':
    app.run(port=8080)

run same ab command, you will get:

Concurrency Level:      200
Time taken for tests:   4.493 seconds
Complete requests:      2000
Failed requests:        0
Total transferred:      330000 bytes
HTML transferred:       24000 bytes
Requests per second:    445.10 [#/sec] (mean)
Time per request:       449.340 [ms] (mean)
Time per request:       2.247 [ms] (mean, across all concurrent requests)
Transfer rate:          71.72 [Kbytes/sec] received

Time per request reduced from 449ms to 247ms!
ref: https://iximiuz.com/en/posts/flask-gevent-tutorial/

Step 1 — Installing the Components
sudo apt install nginx python3-pip python3-dev build-essential libssl-dev libffi-dev python3-setuptools

Step 2 — Creating a Python Virtual Environment
sudo apt install python3-venv
mkdir ~/myproject
cd ~/myproject
python3 -m venv myprojectenv
source myprojectenv/bin/activate

Step 3 — Setting Up a Flask Application
pip install gunicorn flask
nano ~/myproject/myproject.py

from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
    return "<h1 style='color:blue'>Hello There!</h1>"

if __name__ == "__main__":
    app.run(host='0.0.0.0')

Step 4 — Creating the WSGI Entry Point and Configuring Gunicorn
nano ~/myproject/wsgi.py

from myproject import app

if __name__ == "__main__":
    app.run()

cd ~/myproject
gunicorn –bind 0.0.0.0:5000 wsgi:app
then you should be able to open http://your_server_ip:5000

When you have confirmed that it’s functioning properly, press CTRL-C in your terminal window. We’re now done with our virtual environment, so we can deactivate it:
deactivate

Step 5 — Setup Gunicorn service
sudo nano /etc/systemd/system/myproject.service

[Unit]
Description=Gunicorn instance to serve myproject
After=network.target

[Service]
User=ubuntu
Group=ubuntu
WorkingDirectory=/home/ubuntu/myproject
Environment="PATH=/home/ubuntu/myproject/myprojectenv/bin"
ExecStart=/home/ubuntu/myproject/myprojectenv/bin/gunicorn --workers 3 --bind 127.0.0.1:5000 wsgi:app

[Install]
WantedBy=multi-user.target

you can start/enable it and check its status:
sudo systemctl start myproject
sudo systemctl enable myproject
sudo systemctl status myproject

Step 6 — Configuring Nginx to Proxy Requests
sudo nano /etc/nginx/sites-available/myproject

server {
    listen 80;
    server_name ubuntu2020 www.ubuntu2020.ht.home;

    location / {
        include proxy_params;
        proxy_pass http://127.0.0.1:5000;
    }
}

sudo ln -s /etc/nginx/sites-available/myproject /etc/nginx/sites-enabled
sudo nginx -t
sudo systemctl restart nginx
sudo ufw allow ‘Nginx Full’

Step 7 — Securing the Application
 Certbot provides a variety of ways to obtain SSL certificates through plugins. The Nginx plugin will take care of reconfiguring Nginx and reloading the config whenever necessary.
sudo add-apt-repository ppa:certbot/certbot
sudo apt install python-certbot-nginx
sudo certbot –nginx -d your_domain -d www.your_domain
then you can open you site with SSL https://your_domain

Inspired by:
1. How to Create a Self-signed SSL Certificate for Nginx on Ubuntu
2. How to use Flask with gevent WSGI and Gunicorn

import requests

def get_data():
    base_url = 'https://api.github.com/search/repositories?q=language:python+created:%3E2019-12-31&sort=stars&order=desc&per_page=10'
    response = requests.get(base_url)
    result = response.json()
    data = {}
    for item in result['items']:
        #data[item['name']] = [item['html_url'], item['stargazers_count'], item['watchers_count'], item['forks']]
        data[item['name']] = [item['html_url']]
    return data
print(get_data())

PS C:\Users\zhuby> & python c:/Hans/hot_python_githuh.py
{'Depix': ['https://github.com/beurtschipper/Depix'], 'diagrams': ['https://github.com/mingrammer/diagrams'], 'EasyOCR': ['https://github.com/JaidedAI/EasyOCR'], 'avatarify': ['https://github.com/alievk/avatarify'], 'GHunt': 
['https://github.com/mxrch/GHunt'], 'PaddleOCR': ['https://github.com/PaddlePaddle/PaddleOCR'], 'eat_tensorflow2_in_30_days': ['https://github.com/lyhue1991/eat_tensorflow2_in_30_days'], 'yolov5': ['https://github.com/ultralytics/yolov5'], 'Bringing-Old-Photos-Back-to-Life': ['https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life'], 'practical-python': ['https://github.com/dabeaz-course/practical-python']}

1. install library on Ubuntu
   sudo apt install libespeak1
   sudo apt-get install ffmpeg
   pip3 install pdfplumber
   pip3 install pyttsx3
2. prepare the PDF file test.pdf (one page only)

WHAT language is thine, O sea?
   The language of eternal question.
What language is thy answer, O sky?
   The language of eternal silence.

3. write the python3 code read_pdf.py

import pdfplumber
pdf = pdfplumber.open("/home/ubuntu/test.pdf")
print("total pages：",len(pdf.pages))
print("-----------------------------------------")
first_page = pdf.pages[0]
print("current page：",first_page.page_number+1)
print("-----------------------------------------")

text = first_page.extract_text()
print(text)

import pyttsx3
engine = pyttsx3.init()
text = text.replace('\n','')
engine.say(text)
engine.runAndWait()

# save speech as mp3
engine.save_to_file(text, 'test.mp3')
engine.runAndWait()

4. execute the read_pdf.py, get test.mp3