|
home /
infca /
python
(navigation links)
|
De la mà d'en Albert hem arribat a aquest nou mon
|
intro,
venv,
variables,
modules and packages,
modules,
own module
+
how it is found :
*** global access to my modules ***
install,
pip,
go v3 and
update alternatives (select 3.6 or 3.7),
|
rrdtool problem with python3
header,
display version runtime,
parametres, arguments,
quotes,
procedures
|
exceptions
|
logging
|
timestamp
if .. then .. else ,
while loop
|
print()
data types :
lists,
tuples,
sets,
dictionaries
r/w files
|
parse
|
log 2 html
|
PEP8
|
debug
calling python :
from bash,
from node, ....
from python ...
|
numpy &
pandas
TCP/IP,
ping,
FTP
|
opening URLs :
urllib2 or
requests ;
pvoutput requests, by
CURL or
python
|
REST API
server
|
VPN
CSV
|
using JSON
|
matplotlib
*** els meus projectes ***
|
poplib -
imaplib
|
openCV -
ubuntu
|
blog
|
request gLogin()
|
sounds
|
gpio
|
web server
Google Sheets :
mine or
by chatGPT
|
gspread
Google Custom Search :
mine
|
IA/AI :
*** my prompts ***
|
tricks
|
dubtes
|
Links
|
End
|
mapa
|
My first steps into python
go 2 top
Introduction
Python is an interpreted, high-level, generale programming language
Python has a design philosophy that emphasizes code readability, notably using significant whitespace
{wiki}
Python is a strongly typed language
Code sample :
|
sag@odin:~/gpio_python $ cat bucle_sortida_pin37.py
#!/usr/bin/python3
# -*- coding: utf-8 -*-
# executem amb "sudo ./bucle_sortida_pin37.py"
__author__ = "Ivan Grokhotkov"
__version__ = "3.2015.2"
import sys
if sys.version_info[0] == 3: # espressif
from urllib.request import urlretrieve
from urllib.request import urlopen
unicode = lambda s: str(s) # noqa: E731
else:
# not Python 3 - today, it is most likely to be Python 2
from urllib import urlretrieve
from urllib import urlopen
# ===
def my_code( argv ):
'''
Large description
'''
while True:
try:
some_code
except KeyboardInterrupt:
print( "--- kbd Ctrl-C : bye" )
break # exit the "forever" loop
return 0 #
if __name__ == "__main__": # fem un modul reutilitzable
print( "python version", platform.python_version() )
rc = my_code( sys.argv )
sys.exit()
|
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
NOM DEL PROJECTE: [Nom del fitxer]
DESCRIPCIÓ: [Què fa el programa]
AUTOR: [Nom de l'Usuari / Gemini]
DATA: 21 de febrer de 2026
VERSIÓ: 1.2
PROMPT ORIGINAL:
# "[Aquí copiaré el teu prompt]"
"""
import sys
import platform
import os
from datetime import datetime
def my_code(args: list[str]) -> int:
# 1. Identificació del programa i fitxer de log
program_name = os.path.basename(args[0])
log_filename = os.path.splitext(program_name)[0] + ".log"
# 2. Gestió del Log amb context manager
with open(log_filename, "a", encoding="utf-8") as log_file:
# Inici: Timestamp i Paràmetres
start_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
log_file.write(f"[{start_time}] --- INICI EXECUCIÓ ---\n")
log_file.write(f"[{start_time}] Paràmetres: {args[1:]}\n")
# --- ESPAI PER A LA LÒGICA PRINCIPAL ---
# (Aquí anirà el codi que em demanis)
log_file.write(f"[{datetime.now().strftime('%H:%M:%S')}] Executant accions...\n")
# Final: Timestamp de tancament
end_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
log_file.write(f"[{end_time}] --- FINAL EXECUCIÓ ---\n\n")
return 0
if __name__ == "__main__": # fem un mòdul reutilitzable
print( "python version", platform.python_version() )
rc = my_code( sys.argv )
sys.exit(rc)
|
Consells :
The Zen of Python
sebas@minie:~$ python3
Python 3.10.12 (main, Nov 6 2024, 20:22:13) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
>>>
WhirlWinTour
Luís dixit
En mi opinión es un lenguaje muy elegante.
Para hacer programas sencillos es muy agradable de usar.
Pero tiene un problema serio de portabilidad, por dos razones:
- La versión más extendida es la 2, pero hace ya muchos años que la 3 está disponible y tiene muchas mejoras con respecto a la 2.
Por defecto en Linux viene python 2, y por eso cuesta sacárselo de encima.
- El sistema de librerías es un infierno: las dependencias se almacenan de forma global, no en cada proyecto.
Cualquier proyecto que no sea de juguete requiere librerías externas.
Si un proyecto usa una versión de una librería y otro usa otra, entran en conflicto.
Descargar una nueva versión de una librería puede hacer que dejen de funcionar un montón de herramienta que usas en Linux
y no sabías que funcionaban con python.
Para arreglar este problema hay docenas de workarounds
que requieren que el developer tenga que aprender la gestión de "entornos aislados" y cosas por el estilo.
Esto es asumible para un developer voluntarioso,
pero si quieres que tu código pueda ser usado por otros,
les vas a obligar a tener que aprender a usar esos mecanismos de workaround para que también les funcione a ellos.
venv in python
It is highly recommended that you use python virtual environments, or venvs.
A venv is an isolated copy of the python executable and associated files,
which provides a separate environment for installing python modules, independent from the system python installation.
You can have as many venvs as you need, which means that you can have multiple module configurations set up,
and you can switch between them easily.
Read realpython intro
sebas@minie:~/dades/Cifras_y_Letras$ python3 -m venv env && source ./env/bin/activate
(env) sebas@minie:~/dades/Cifras_y_Letras$ python3 -m pip install -U pip wheel setuptools
wget https://files.pythonhosted.org/packages/04/8a/d336e6f73bd988630488c3d6b9eb689c360f53ea0751893156ba77591e52/countdown_numbers_solver-1.0.0-py3-none-any.whl
(env) sebas@minie:~/dades/Cifras_y_Letras$ pip install countdown_numbers_solver-1.0.0-py3-none-any.whl
Processing ./countdown_numbers_solver-1.0.0-py3-none-any.whl
Installing collected packages: countdown-numbers-solver
Successfully installed countdown-numbers-solver-1.0.0
(env) sebas@minie:~/dades/Cifras_y_Letras$
env) sebas@minie:~/dades/Cifras_y_Letras$ python
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import countdown_numbers_solver
>>> countdown_numbers_solver.solve([ 100, 25, 8, 3, 1, 1 ], 984)
( 100 + 25 + 1 - 3 ) x 8
( 100 + 25 - 1 - 1 ) x 8
( 100 + 1 ) x ( 8 + 1 ) + 25 x 3
( ( 100 - 1 ) / 3 + 8 ) x ( 25 - 1 )
Total: 4 solutions.
>>>
>>> countdown_numbers_solver.solve([ 1, 2, 3, 4, 5, 6 ], 279)
( 6 x 5 + 1 ) x ( 4 + 3 + 2 )
( 6 x ( 3 + 2 ) + 1 ) x ( 5 + 4 )
( 6 + 2 ) x ( 4 + 3 ) x 5 - 1
( ( 4 + 2 ) x 5 + 1 ) x ( 6 + 3 )
Total: 4 solutions.
>>> countdown_numbers_solver.solve([ 7, 4, 4, 9, 9, 2 ], 958)
Total: 0 solutions.
(venv) $ deactivate
$
Another project :
- create a new directory for your project and navigate into it.
- set up a virtual environment for Python dependencies:
python3 -m venv venv
source venv/bin/activate
- install Flask and Flask components :
pip install Flask Flask-SQLAlchemy Flask-Login flask-wtf email_validator
Flask
python variables
In many programming languages, variables are best thought of as containers or buckets into which you put data.
In Python, by contrast, variables are best thought of not as containers but as pointers.
one consequence of this: because Python variables just point to various objects, there is no need to "declare" the variable, or even require the variable to always point to information of the same type!
x = 1 # x is an integer
x = 'hello' # now x is a string
x = [1, 2, 3] # now x is a list
There is a consequence of this "variable as pointer" approach that you need to be aware of.
If we have two variable names pointing to the same mutable object, then changing one will change the other as well!
WhirlWindTour ch.03
python types - Built-In Data Structures
Python's simple types are summarized in the following table:
** Python Scalar Types **
Type Example Description
----- -------- ------------
int x = 1 integers (i.e., whole numbers)
float x = 1.0 floating-point numbers (i.e., real numbers)
complex x = 1 + 2j Complex numbers (i.e., numbers with real and imaginary part)
bool x = True Boolean: True/False values
str x = 'abc' String: characters or text
NoneType x = None Special object indicating nulls
Python also has several built-in compound types, which act as containers for other types. These compound types are:
** Python Compound Types **
Type Example Description
----- -------- ------------
list [1, 2, 3] Ordered collection
tuple (1, 2, 3) Immutable ordered collection
dict {'a':1, 'b':2, 'c':3} Unordered (key,value) mapping
set {1, 2, 3} Unordered collection of unique values
WhirlWindTour ch.06
python modules and packages
Modules and packages are two mechanisms that facilitate modular programming.
Any python file is a module, its name being the file's base name without the .py extension.
A package is a collection of python modules: while a module is a single python file,
a package is a directory of python modules containing an additional __init__.py file.
stack overflow
A package or module name can *not* start with a number -
stackoverflow -
it will trigger a SyntaxError
There are three different ways to define a module in python:
- a module can be written in python itself.
- a module can be written in C and loaded dynamically at run-time, like the re (regular expression) module.
- a built-in module is intrinsically contained in the interpreter, like the itertools module.
real python modules and packages {***}
A module is a simple python file that contains collections of functions and global variables and with having a .py extension file.
A package is a directory having python modules and also having a __init__.py file.
geeks 4 geeks
"dir(module)" returns the names of the attributes of the "module" (list of methods, properties and submodules)
dir() does not list the names of built-in functions and variables.
If you want a list of those, they are defined in the standard module builtins:
>>> import builtins
>>> dir(builtins)
For all the details of a module, try "help(module_name)"
python modules
A module is a file containing python definitions and statements.
The file name is the module name with the suffix .py appended.
The definitions from a module can be imported into other modules or into the main module.
import statement
The statement
import fibo
... does not enter the names of the functions in the current symbol table ;
it only enters the module name.
Using the module name you can access the functions :
>>> fibo.fib2(1000)
If you intend to use a function often you can assign it a local name :
>>> myfib = fibo.fib2
>>> myfib(1000)
There is a variant of the import statement that imports names from a module directly into the importing module’s symbol table.
>>> from fibo import fib2, fib3
>>> fib3(500)
This does not introduce the module name from which the imports are taken in the local symbol table,
so in the example, fibo is not defined.
If the module name is followed by as, then the name following as is bound directly to the imported module.
>>> import fibo as fibi
>>> fibi.fib2(500)
This is effectively importing the module in the same way that import fibo will do,
with the only difference of it being available as fibi.
It can also be used when using from with similar effects:
>>> from fibo import fib2 as fibonacci
>>> fibonacci(500)
The important one for me is :
sebas@pi0alby:~/python/bot $ cat veure_versio.py
#!/usr/bin/python3
from telebot import version as my_versio_tb
print( "(0) tb version", my_versio_tb.__version__ )
The Module Search Path
Let’s take a look at what happens when python executes the statement:
When the interpreter executes the above import statement, it searches for mod.py in a list of directories assembled from the following sources:
- The directory from which the input script was run or the current directory if the interpreter is being run interactively
- The list of directories contained in the PYTHONPATH environment variable, if it is set. (The format for PYTHONPATH is OS-dependent but should mimic the PATH environment variable.)
- An installation-dependent list of directories configured at the time Python is installed
The resulting search path is accessible in the Python variable sys.path, which is obtained from a module named sys:
>>> import sys
>>> sys.path
['', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload',
'/usr/local/lib/python3.10/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.10/dist-packages']
>>>
good "import" sample
Goal : we want to have some config values in a file, included in .gitignore
- in main progarm "bot.py", we code
import config # config.py holds bot token as "bot_token"
- we specify the configuration values :
nicolau@mars:~/sebas/python/telegram/telegram.ext/smokey$ cat config.py
bot_token='1234567890:AABBCCDDEEffggxQYV3wQqDLL8zNfrJwPot'
- finally we use them in main code (bot.py)
nicolau@mars:~/sebas/python/telegram/telegram.ext/smokey$ cat bot.py
updater = Updater( config.bot_token, workers=10, use_context=True )
own project with own import file
Al fitxer arrel, posem :
pi@R4:~/python/pkw/bin $ cat pkw_cli.py
import pkw_pkg
I al directori hi tenim :
pi@R4:~/python/pkw/bin $ ls pkw_pkg/
4 -rw-r--r-- 1 pi pi 118 May 7 22:23 __init__.py
4 -rw-r--r-- 1 pi pi 1614 May 13 21:44 pkw_graph.py
4 -rw-r--r-- 1 pi pi 1525 May 16 22:01 pkw_initialize.py
4 -rw-r--r-- 1 pi pi 3034 May 16 22:03 pkw_reader.py
Ara al fitxer principal podem fer servir les funcions implementades :
pkw_pkg.pkw_initialize.create_rrd(rrd_filename, overwrite=True)
t, h, p = pkw_pkg.pkw_reader.defronius(simulate=simulate)
pkw_pkg.pkw_graph.fer_png_3_valors ( rrd_filename,
*** how and where to define own package in own module ***
... a package is a directory of python modules containing an additional __init__.py file (even empty)
sebas@minie:~/dades/python/pkw/eines$ python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
>>> from newp import bubu # from package import module
>>> bubu.vfy() # call module.function (including parenthesis !)
*** verificar file fn.txt ***
True
>>> print(bubu.s) # display string defined in module
un string definit a BUBU
sebas@minie:~/dades/python/pkw/eines$ ls
newp
sebas@minie:~/dades/python/pkw/eines/newp$ ls
bubu.py __init__.py
sebas@minie:~/dades/python/pkw/eines/newp$ cat bubu.py
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
s = 'un string definit a BUBU'
def vfy ( fnCSV = 'fn.txt', bVerbose = True ) :
bRC = True
print ( '*** verificar file {} ***'.format(fnCSV) )
return bRC
real python {***}
trace "import" sentences
I have one py that finds RF24 and another that does not
sebas@pi0alby:~/nrf24l01/nrf24 $ ./1_getting_started.py
from RF24 import RF24, RF24_PA_LOW
ImportError: No module named RF24
sebas@pi0alby:~/nrf24l01/rf24libs/RF24/examples_linux $ python3 ./getting_started.py
Traceback (most recent call last):
RuntimeError: can't open device
How to know how python finds a module ?
We could have
ImportError: No module named <module name>
Anem al directori i fem
$ python3
import sys
sys.path
As :
pi@R4:~/python $ python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload',
'/home/pi/.local/lib/python3.7/site-packages', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages']
>>>
Standard package ubication
The package must be one directory bellow your code :
nicolau@mars:~/sebas/python/fronius$ mkdir pkg_fronius
nicolau@mars:~/sebas/python/fronius$ touch pkg_fronius/__init__.py
nicolau@mars:~/sebas/python/fronius$ python3
Python 3.7.5 (default, Dec 9 2021, 17:04:37)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_fronius
>>>
Gracies, Albert !
I el seu contingut :
pi@pi0:~/python/fronius/pkg_fronius $ cat from_fronius_json_extract_dict.py
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import json
# =============== own functions
def get_dictionary( szFN ):
print( "(pkg) >>> get a dictionary from file {", szFN, "}" )
try:
pf = open( szFN, 'r' )
dt = json.load( pf ) # read json from file into dictionary
pf.close()
my_body = dt['Body']
my_values = my_body['Data']['inverter/1']['Data']['Temperature_Powerstage']['Values']
...
return my_dict
Comprovem importacio :
pi@pi0:~/python/fronius $ python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44)
>>> import pkg_fronius
>>> print(pkg_fronius)
<module 'pkg_fronius' from '/home/pi/python/fronius/pkg_fronius/__init__.py'>
>>> print(dir(pkg_fronius))
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__']
>>>
Com el fem servir ?
pi@pi0:~/python/fronius $ cat y_read_temperatures_and_plot_their_max.py
import pkg_fronius
rd = pkg_fronius.from_fronius_json_extract_dict.get_dictionary()
own packages - global access
{20220806} Has de copiar tots els teus moduls al directori de llibreries de python, al subdirectory que mes t'agradi exemple "sebas".
Llavors podras fer un "import sebas" des de qualsevol python.
- ens aclarim amb quin python treballem i el seu directori :
pi@pi0:~ $ python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44)
>>> import sys
>>> sys.path
['', '/usr/lib/python37.zip', '/usr/lib/python3.7', ...
- hi fem un directori per les nostres eines :
pi@pi0:/usr/lib/python3.7 $ sudo mkdir sebas_eines
- fem que sigui un package :
pi@pi0:/usr/lib/python3.7 $ sudo touch sebas_eines/__init__.py
pi@pi0:/usr/lib/python3.7/sebas_eines $ ls
0 -rw-r--r-- 1 root root 0 Aug 6 11:46 __init__.py
- fem proves de "import" :
pi@pi0:~ $ python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sebas_eines
- codifiquem un servei :
pi@pi0:/usr/lib/python3.7/sebas_eines $ cat greeting.py
def say_hello(name: str) -> str:
return f'Hello {name}!'
- codifiquem un client :
pi@pi0:~/python/proves $ cat mestre.py
#!/bin/python3
from sebas_eines import greeting
rc = greeting.say_hello("Sebas")
print( rc )
- verifiquem q funciona :
pi@pi0:~/python/proves $ ./mestre.py
Hello Sebas!
Executing modules as scripts
When you run a python module with ( url )
python fibo.py <arguments>
... the code in the module will be executed, just as if you imported it,
but with the "__name__" set to "__main__".
That means that by adding this code at the end of your module:
if __name__ == "__main__":
import sys
fib(int(sys.argv[1]))
... you can make the file usable as a script as well as an importable module,
because the code that parses the command line only runs if the module is executed as the “main” file
Again : the __name__ test at the bottom is true only when this file is run as a top-level script (e.g., from a command line, via an icon click, or within the IDLE GUI),
not when it is imported.
display module version
There is no "standard" way :
print( platform.python_version() ) # python
print( "NumPy version %s" % numpy.__version__ ) # numerical python
print( "CV2 version is " + cv2.__version__ ) # cv2
print( "SYS version is " + str(sys.version_info[0]) + "." + str(sys.version_info[1]) + "." + str(sys.version_info[2]) )
$ python -c "import requests; print( requests.__version__ )"
$ python -c 'from telebot import version; print( "telebot version is", version.__version__ )'
r0 $ python3 -c 'import matplotlib; print(matplotlib.__version__, matplotlib.__file__)'
display module contents - telebot
from here :
sebas@pi0alby:~/python/bot $ python
Python 3.7.3 (default, Dec 20 2019, 18:57:59)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import telebot
>>> dir(telebot)
['AsyncTeleBot', 'FileHandlerBackend', 'Handler', 'MemoryHandlerBackend', 'TeleBot', '__builtins__', '__cached__',
'__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'apihelper',
'console_output_handler', 'formatter', 'handler_backends', 'logger', 'logging', 'print_function',
're', 'six', 'sys', 'threading', 'time', 'types', 'util']
>>> dir(telebot.TeleBot)
['_TeleBot__non_threaded_polling', '_TeleBot__notify_update', '_TeleBot__retrieve_updates',
'_TeleBot__skip_updates', '_TeleBot__threaded_polling', '__class__', '__delattr__', '__dict__',
'__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__',
'__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__',
'__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__',
'__weakref__', '_build_handler_dict', '_exec_task', '_notify_command_handlers', '_notify_next_handlers',
'_notify_reply_handlers', '_test_filter', '_test_message_handler', 'add_callback_query_handler',
'add_channel_post_handler', 'add_chosen_inline_handler', 'add_edited_channel_post_handler',
'add_edited_message_handler', 'add_inline_handler', 'add_message_handler', 'add_middleware_handler',
'add_poll_answer_handler', 'add_poll_handler', 'add_pre_checkout_query_handler', 'add_shipping_query_handler',
'add_sticker_to_set', 'answer_callback_query', 'answer_inline_query', 'answer_pre_checkout_query',
'answer_shipping_query', 'callback_query_handler', 'channel_post_handler', 'chosen_inline_handler',
'clear_reply_handlers', 'clear_reply_handlers_by_message_id', 'clear_step_handler',
'clear_step_handler_by_chat_id', 'create_new_sticker_set', 'delete_chat_photo',
'delete_chat_sticker_set', 'delete_message', 'delete_sticker_from_set', 'delete_webhook',
'disable_save_next_step_handlers', 'disable_save_reply_handlers', 'download_file',
'edit_message_caption', 'edit_message_live_location', 'edit_message_media', 'edit_message_reply_markup',
'edit_message_text', 'edited_channel_post_handler', 'edited_message_handler', 'enable_save_next_step_handlers',
'enable_save_reply_handlers', 'export_chat_invite_link', 'forward_message', 'get_chat',
'get_chat_administrators', 'get_chat_member', 'get_chat_members_count', 'get_file', 'get_file_url',
'get_game_high_scores', 'get_me', 'get_sticker_set', 'get_updates', 'get_user_profile_photos', 'get_webhook_info',
'infinity_polling', 'inline_handler', 'kick_chat_member', 'leave_chat', 'load_next_step_handlers',
'load_reply_handlers', 'message_handler', 'middleware_handler', 'pin_chat_message', 'poll_answer_handler',
'poll_handler', 'polling', 'pre_checkout_query_handler', 'process_middlewares', 'process_new_callback_query',
'process_new_channel_posts', 'process_new_chosen_inline_query', 'process_new_edited_channel_posts',
'process_new_edited_messages', 'process_new_inline_query', 'process_new_messages', 'process_new_poll',
'process_new_poll_answer', 'process_new_pre_checkout_query', 'process_new_shipping_query', 'process_new_updates',
'promote_chat_member', 'register_for_reply', 'register_for_reply_by_message_id', 'register_next_step_handler',
'register_next_step_handler_by_chat_id', 'remove_webhook', 'reply_to', 'restrict_chat_member',
'send_animation', 'send_audio', 'send_chat_action', 'send_contact', 'send_dice', 'send_document', 'send_game',
'send_invoice', 'send_location', 'send_media_group', 'send_message', 'send_photo', 'send_poll', 'send_sticker',
'send_venue', 'send_video', 'send_video_note', 'send_voice', 'set_chat_administrator_custom_title',
'set_chat_description', 'set_chat_permissions', 'set_chat_photo', 'set_chat_sticker_set', 'set_chat_title',
'set_game_score', 'set_my_commands', 'set_sticker_position_in_set', 'set_update_listener', 'set_webhook',
'shipping_query_handler', 'stop_bot', 'stop_message_live_location', 'stop_poll', 'stop_polling',
'unban_chat_member', 'unpin_chat_message', 'upload_sticker_file']
A bit better :
#!/usr/bin/python3
import telebot
print( telebot.__path__ )
['/usr/local/lib/python3.7/dist-packages/telebot']
display module contents - requests
Sembla que amb la versio 2.7 podem fer servir requests.exceptions.HTTPError ...
Revisem la documentacio !
pi@R4:~/python/fronius $ python
Python 2.7.16 (default, Oct 10 2019, 22:02:15)
[GCC 8.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> print(dir(requests))
['ConnectTimeout', 'ConnectionError', 'DependencyWarning', 'FileModeWarning', 'HTTPError', 'NullHandler', 'PreparedRequest', 'ReadTimeout',
'Request', 'RequestException', 'RequestsDependencyWarning', 'Response', 'Session', 'Timeout', 'TooManyRedirects', 'URLRequired',
'__author__', '__author_email__', '__build__', '__builtins__', '__cake__', '__copyright__', '__description__', '__doc__',
'__file__', '__license__', '__name__', '__package__', '__path__', '__title__', '__url__', '__version__',
'_check_cryptography', '_internal_utils', 'adapters', 'api', 'auth', 'certs', 'chardet', 'check_compatibility', 'codes', 'compat', 'cookies',
'cryptography_version', 'delete', 'exceptions', 'get', 'head', 'hooks', 'logging', 'models', 'options', 'packages', 'patch', 'post', 'put',
'pyopenssl', 'request', 'session', 'sessions', 'status_codes', 'structures', 'urllib3', 'utils', 'warnings']
>>> print(dir(requests.exceptions))
['BaseHTTPError', 'ChunkedEncodingError', 'ConnectTimeout', 'ConnectionError', 'ContentDecodingError', 'FileModeWarning',
'HTTPError', 'InvalidHeader', 'InvalidProxyURL', 'InvalidSchema', 'InvalidURL', 'MissingSchema', 'ProxyError',
'ReadTimeout', 'RequestException', 'RequestsDependencyWarning', 'RequestsWarning', 'RetryError', 'SSLError',
'StreamConsumedError', 'Timeout', 'TooManyRedirects', 'URLRequired', 'UnrewindableBodyError',
'__builtins__', '__doc__', '__file__', '__name__', '__package__']
sebas@pi0alby:~/python/bot $ python
Python 3.7.3 (default, Dec 20 2019, 18:57:59)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> print(dir(requests))
['ConnectTimeout', 'ConnectionError', 'DependencyWarning', 'FileModeWarning', 'HTTPError', 'NullHandler', 'PreparedRequest', 'ReadTimeout',
'Request', 'RequestException', 'RequestsDependencyWarning', 'Response', 'Session', 'Timeout', 'TooManyRedirects', 'URLRequired',
'__author__', '__author_email__', '__build__', '__builtins__', '__cached__', '__cake__', '__copyright__', '__description__', '__doc__',
'__file__', '__license__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__title__', '__url__', '__version__',
'_check_cryptography', '_internal_utils', 'adapters', 'api', 'auth', 'certs', 'chardet', 'check_compatibility', 'codes', 'compat', 'cookies',
'cryptography_version', 'delete', 'exceptions', 'get', 'head', 'hooks', 'logging', 'models', 'options', 'packages', 'patch', 'post', 'put',
'pyopenssl', 'request', 'session', 'sessions', 'status_codes', 'structures', 'urllib3', 'utils', 'warnings']
>>> print(dir(requests.ConnectTimeout))
['__cause__', '__class__', '__context__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__',
'__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__',
'__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__',
'__suppress_context__', '__traceback__', '__weakref__', 'args', 'characters_written', 'errno', 'filename', 'filename2', 'strerror', 'with_traceback']
display module version - requests
We can use importlib.metadata since python 3.8 :
print( "(4) version telebot estatica" )
# python >=3.8
from importlib.metadata import version
version('telebot')
Yep ! Went into importlib_metadata
python "class" and "self" and "__init__" methods
- self
The word 'self' is used to represent the instance of a class. By using the "self" keyword we access the attributes and methods of the class in python.
- __init__ method
"__init__" is a reserved method in python classes.
It is called as a constructor in object oriented terminology.
This method is called when an object is created from a class and it allows the class to initialize the attributes of the class.
url
split python code into modules
Lets have "main" file demo.py :
#!/usr/bin/python3
# -*- coding: utf-8 -*-
'''
Demo for the TP-Link A19-LB130 RBGW WiFi bulb
'''
import sys
import time
from tplight import LB130
def my_code( argv ):
'''
Main program function
'''
light = LB130("10.0.0.130") # create an instance of the light with its IP address
if __name__ == "__main__":
my_code( sys.argv )
sys.exit()
The secondary file is
tplight.py :
#!/usr/bin/python3
# -*- coding: utf-8 -*-
'''
Control class for TP-Link A19-LB130 RBGW WiFi bulb
'''
import datetime
import socket
import json
import sys
class LB130(object):
'''
Methods for controlling the LB130 bulb
'''
encryption_key = 0xAB
__udp_ip = "10.0.0.130"
calling python from python without modules
os.system('python calendario.py')
subprocess.getoutput('python3 nom.py')
python versions
python v2 (2.6) vs v3 (3.6) main differences
Very few :
- "print" statement
The print statement has been replaced with a print() function - details
- libraries - old libraries built for python 2 are not forward compatible
But new libraries are created strictly for python 3
- strings are stored : ASCII (v2) or Unicode (v3)
- division : v2 rounds down (by default), v3 does not round
python.org docs,
url
some more differences I've found
- //pi0/home/sebas/python/mostrar_temperatura.py
old code
import commands #
tempgpu = commands.getoutput('/opt/vc/bin/vcgencmd measure_temp' ).replace('temp=', '' )
new code
from subprocess import check_output
tempgpu = check_output(["vcgencmd","measure_temp"]).decode()
how to format print()
if err:
my_node_version = '--- %s ---' % [err]
szName = "My name is {}, I'm {}".format( "John", 36 )
>>> v0, v1, v2 = "a", "b", "c"
>>> print("%s%s%s" % (v0, v1, v2))
abc
>>> print("%s%s%s".format(v0, v1, v2)
abc
>>> print(f"{v0}{v1}{v2}")
abc
From an improved string formatting syntax Guide :
%-formatting - name = "Eric" ; age = 74 ; "Hello, %s. You are %s." % (name, age)
f-Strings - f"Hello, {name}. You are {age}."
str.format() - "Hello, {}. You are {}.".format(name, age)
print( 'Tot {:.2f}'.format (ftot) ) - 2 decimals only
f-strings are expressions evaluated at runtime rather than constant values !
If we want to provide formatting information :
val = 12.3
print(f'{val:.2f}') # 12.30
print(f'{val:.5f}') # 12.30000
zetcode : width, justify, numeric notations,
strings and bytes literals
python 3.x makes a clear distinction between the types:
- str = '...' literals = a sequence of Unicode characters (UTF-16 or UTF-32, depending on how python was compiled)
- bytes = b'...' literals = a sequence of octets (integers between 0 and 255)
Bytes literals are always prefixed with 'b' or 'B'; they produce an instance of the bytes type instead of the str type.
You use "str" when you want to represent text.
You use "bytes" when you want to represent low-level binary data like structs.
You can encode a str to a bytes object :
>>> '\uFEFF'.encode('UTF-8')
b'\xef\xbb\xbf'
And you can decode a bytes into a str :
>>> b'\xE2\x82\xAC'.decode('UTF-8')
'€'
docs.python.org
Unicode literal strings
Python 2 uses two different types for strings, unicode and str.
When you define a literal string the interpreter actually creates a new object of type str that holds this literal.
s = "A literal string"
print type(s)
<type 'str'>
You can explicitly define a literal string as unicode using the u prefix.
The interpreter will create instead a unicode object for this literal.
s = u"A literal string"
print type(s)
<type 'unicode'>
For ASCII text, using str type is sufficient but if you intend to manipulate non-ASCII text it is important to use unicode type for character level operations to work correctly.
stackoverflow
Unicode details
docs.python {****}
installing python packages
Some python packages can be found in the Raspbian archives, and can be installed using apt, for example:
sudo apt update
sudo apt install python-picamera
Not all python packages are available in the Raspbian archives, and those that are can sometimes be out of date.
If you can't find a suitable version in the Raspbian archives,
you can install packages from the
Python Package Index (known as PyPI).
To do so, install pip:
sudo apt-get install python-pip {python v2}
sudo apt-get install python3-pip {python v3}
Then install python packages (e.g. simplejson) with pip3:
pip3 install simplejson
pip3 install ifaddr
python3 -m pip install ifaddr
It is also easy to remove them :
python3 -m pip install textparser
python3 -m pip uninstall textparser
And to list the installed packages :
$ sudo pip3 list
$ apt list --installed
$ dpkg -l
Sometimes we need to upgrade a package :
$ python -m pip install --upgrade pip
Sometimes, if you dont use "sudo", you get this ugly message :
nicolau@mars:~$ pip3 list
Traceback (most recent call last):
File "/home/nicolau/.local/bin/pip3", line 5, in <module>
from pip._internal.cli.main import main
ModuleNotFoundError: No module named 'pip._internal'
Also on install :
nicolau@mars:~$ pip install Cython
Traceback (most recent call last):
File "/home/nicolau/.local/bin/pip", line 5, in
from pip._internal.cli.main import main
ModuleNotFoundError: No module named 'pip._internal'
nicolau@mars:~$ sudo pip install Cython
If you want to run the service as root, you have to install the module with sudo: sudo python3 -m pip install my_module
repositories and PPAs
Personal Package Archives (PPAs) are software repositories designed for Ubuntu users and are easier to install than other third-party repositories.
$ /etc/apt/sources.list
$ cat /etc/apt/sources.list.d
how to use pip
- Dont run pip under "sudo" !
url :
when you run pip with sudo, you run setup.py with sudo, so you run arbitrary python code from the internet as root
- Always launch it from python :
$ python -m pip install <package name>
- install a specific version :
nicolau@mars:~$ python -m pip install pyqt5==5.15.1
- install requirements in a file :
nicolau@mars:~$ python -m pip install -r requirements.txt
nicolau@mars:~$ cat requirements.txt
PyQt5==5.15.1
tuyapy==0.1.3
requests==2.24.0
- verify your installation :
nicolau@mars:~$ python -m pip check
launchpadlib 1.10.6 requires testresources, which is not installed.
telebot 0.0.3 has requirement requests==2.7.0, but you have requests 2.24.0.
resum i verify
The correct form is
python3 -m pip install package_name
Si pip ja apunta a python3 llavors podem
pip install requests
Per comprovar on apunta pip :
sebas@minie:~$ pip --version
pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
sebas@minie:~$ pip3 --version
pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
sebas@minie:~$ python3 -m pip --version
pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
Under guindows
[20160604] Install under W2008R2 at W500.
Compte : instalar python
v2 :
2.7.11 - normal "Windows x86 MSI installer"
Llegir
Using Windows
Resultat :
C:\Python27> python --version
Python 2.7.11
Using PIP installer under windows
If a name.py code contains a "import" sentence, this "code.py" must be at this directory : "c:\python27\lib"
You can install a module using "c:\python27\scripts\pip install modulename"
C:\Python27\Scripts> .\pip install pyping
Collecting pyping
Downloading pyping-0.0.5.tar.gz
Installing collected packages: pyping
Running setup.py install for pyping
Successfully installed pyping-0.0.5
You are using pip version 7.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
Lets update "pip" :
C:\> python -m pip install --upgrade pip
Collecting pip
Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB)
100% |################################| 1.3MB 292kB/s
Installing collected packages: pip
Found existing installation: pip 7.1.2
Uninstalling pip-7.1.2:
Successfully uninstalled pip-7.1.2
Successfully installed pip-9.0.1
Lets visit "pyping" again :
C:\> python -m pip install --upgrade pyping
Requirement already up-to-date: pyping in c:\python27\lib\site-packages
Use pyping sample.
python on Raspberry
Read about installing software in python
First steps
In my Raspberry we have :
$ python --version
Python 2.7.13
$ which python
/usr/bin/python
$ python2 --version
Python 2.7.13
$ which python2
/usr/bin/python2
$ python3 --version
Python 3.5.3
$ which python3
/usr/bin/python3
How does linux know which one to use ?
sebas@henry:~ $ ls -l $(which python)
lrwxrwxrwx 1 root root 9 Jan 24 2017 /usr/bin/python -> python2.7
To change it, use
sebas@henry:~ $ sudo ln -sfv /usr/bin/python3 $(which python)
update pip under linux
python -m pip install --upgrade pip
Installing packages
I've done :
- python -m pip install psutil - display CPU use, etc
- sudo pip3 install psutil - python v3
moving to python v3
Version we have :
nicolau@mars:~ $ python --version
Python 3.6.9
Problems and solutions :
- speedtest : update
- pip : sudo apt-get install python3-pip
- numpy - ModuleNotFoundError: No module named 'numpy'
python -m pip install --upgrade numpy
- cv2 - ModuleNotFoundError: No module named 'cv2' : sudo pip install opencv-python
url : python -m pip install opencv-python
We do it on :
- 202006 - MARS/Ubuntu
nicolau@mars:~$ python --version
Python 3.6.9
20201204 - we need "python-kasa", which requires "Python >=3.7, <4.0"
- update whole system
nicolau@mars:~$ sudo apt update -y
- install v 3.7
nicolau@mars:~$ sudo apt install python3.7
- add python 3.6 & python 3.7 to update-alternatives
$ sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 1
update-alternatives: using /usr/bin/python3.6 to provide /usr/bin/python3 (python3) in auto mode
$ sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.7 2
update-alternatives: using /usr/bin/python3.7 to provide /usr/bin/python3 (python3) in auto mode
- update python 3 to point to python 3.7
nicolau@mars:~$ sudo update-alternatives --config python3
There are 2 choices for the alternative python3 (providing /usr/bin/python3).
Selection Path Priority Status
------------------------------------------------------------
* 0 /usr/bin/python3.7 2 auto mode
1 /usr/bin/python3.6 1 manual mode
2 /usr/bin/python3.7 2 manual mode
Press <enter> to keep the current choice[*], or type selection number:
- display actual version :
nicolau@mars:~$ python --version
Python 3.7.5
url
"llegir_preu_electre.py" li cal v 3.7+ (see)
nicolau@mars:~$ which python
/usr/bin/python
nicolau@mars:~$ ls -al /usr/bin/python
0 lrwxrwxrwx 1 root root 16 May 11 2020 /usr/bin/python -> /usr/bin/python3
nicolau@mars:~$ ls -al /usr/bin/python3
0 lrwxrwxrwx 1 root root 25 Dec 4 2020 /usr/bin/python3 -> /etc/alternatives/python3
nicolau@mars:~$ ls -al /etc/alternatives/python3
0 lrwxrwxrwx 1 root root 18 Jan 31 13:39 /etc/alternatives/python3 -> /usr/bin/python3.7
Compte : python 3.7 breaks apt-get : solution
Now we come up with a very interesting problem : "term" does no open
"syslog" has a message like
nicolau@mars:~$ cat /var/log/syslog
ImportError: cannot import name '_gi' from 'gi' (/usr/lib/python3/dist-packages/gi/__init__.py)
Solution (can't open a Terminal !) :
- navegar pel filesystem "grafic" i picar en qualsevol carpeta "Open in Terminal"
- corregir
sudo gedit /usr/bin/gnome-terminal
... change
#!/usr/bin/python3 to #!/usr/bin/python3.6
Another problem : red circle with a white minus sign in the toolbar + "A problem occurred when checking for the updates"
sudo update-alternatives --config python3
- set it to 3.6
- use "/usr/bin/python3.7" in your code
nicolau@mars:~/sebas/python/smart_plug$ cat plug_power.py
#!/usr/bin/python3.7
sebas@minie:~/dades/python/smart_devices$ more 1_plug_power.py
#!/usr/bin/python3
- 20200710 - R3 - go 3.7 :
- download and extract
pi@ai-pi:~ $ sudo su
root@ai-pi:/home/pi# cd /usr/src
root@ai-pi:/usr/src# wget https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tgz
root@ai-pi:/usr/src# tar -xf Python-3.7.0.tgz
- install dependencies
sudo apt-get update
sudo apt-get install -y build-essential tk-dev libncurses5-dev libncursesw5-dev libreadline6-dev
libdb5.3-dev libgdbm-dev libsqlite3-dev libssl-dev libbz2-dev libexpat1-dev
liblzma-dev zlib1g-dev libffi-dev
- configure and install python3 :
root@ai-pi:/usr/src# cd /usr/src/Python-3.7.0
root@ai-pi:/usr/src/Python-3.7.0# ./configure --enable-optimizations { *** wow *** }
root@ai-pi:/usr/src/Python-3.7.0# make altinstall { *** wow wow 15 min *** }
- update the links
root@ai-pi:/usr/src# ln -s /usr/local/bin/python3.7 /usr/local/bin/python3
- edit "/usr/bin/lsb_release" to have "#!/usr/bin/python2.7"
- verify versions
root@odin:~# python --version
Python 3.4.2
root@odin:~# python3 --version
Python 3.7.0
sag@odin:/usr/local/bin $ pip3.7 -V
pip 10.0.1 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)
sag@odin:~ $ pkg-config --libs openssl
-lssl -lcrypto
sag@odin:~ $ apt list *certi*
Listing... Done
ca-certificates/oldstable,now 20200601~deb9u1 all [installed]
ca-certificates-java/oldstable 20170929~deb9u3 all
ca-certificates-mono/oldstable,now 4.6.2.7+dfsg-1 all [installed,automatic]
fusiondirectory-plugin-certificates/oldstable 1.0.19-1+deb9u1 all
python-certifi/oldstable 2016.2.28-1 all
python3-certifi/oldstable 2016.2.28-1 all
ruby-certificate-authority/oldstable 0.2.0~6dd483bf-1 all
xul-ext-certificatepatrol/oldstable 2.0.14-5 all
You can revert back to the default python 3 version anytime by simply updating the link created above.
Also keep in mind updating raspbian will likely break this and you may have to recreate the links if needed.
20201110, R3 - go back to v 3.5
sag@odin:/usr/bin $ sudo rm /usr/local/bin/python3
sag@odin:/usr/bin $ which python3
/usr/bin/python3
sag@odin:/usr/bin $ python3 -V
-bash: /usr/local/bin/python3: No such file or directory
ln -s /usr/bin/python3 /usr/local/bin/python3
sag@odin:~ $ python3 -V
Python 3.5.3
- sudo apt install libssl-dev libncurses5-dev libsqlite3-dev libreadline-dev libtk8.5 libgdm-dev libdb4o-cil-dev libpcap-dev
- sudo apt-get install python3.5 : "Unable to locate package python3.5"
- 20201025 - R0 -
url :
sebas@pi0alby:~ $ sudo apt-get update
$ sudo apt install python3 idle3
$ sudo apt autoremove
$ sudo apt install python3-pip
sebas@pi0alby:~ $ python -V
Python 2.7.16
sebas@pi0alby:~ $ python3 -V
Python 3.7.3
Cridem a "python3" explicitament en el codi !
Instalem pip3 amb "python3 -m pip install --upgrade pip"
Instalem mitjançant pip3 install --user <package_name>
SSL problem
On almost any install using pip3 or python3 (requests on R3), I get :
sag@odin:~ $ python3 -m pip install --upgrade pip
pip is configured with locations that require TLS/SSL, however the ssl module in python is not available.
OpenSSL versions are :
sag@odin:/usr/local/bin $ python
Python 2.7.13 (default, Sep 26 2018, 18:42:22)
[GCC 6.3.0 20170516] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.1.0l 10 Sep 2019'
>>>
sag@odin:/usr/local/bin $ python3
Python 3.7.0 (default, Jul 10 2020, 18:31:19)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/ssl.py", line 98, in <module>
import _ssl # if we can't import it, let the error propagate
ModuleNotFoundError: No module named '_ssl'
>>>
After moving back from 3.7 to 3.5 (see above), ssl works fine :
sag@odin:~ $ python3
Python 3.5.3 (default, Sep 27 2018, 17:25:39)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.1.0l 10 Sep 2019'
>>>
RRDTOOL problem with python3
*** Cant use rrdtool with python3 ***
why ?
pi@R4:~/python/pkw/bin $ cat pkw_cli.py
#!/usr/bin/python2
#
# amb python3 tenim
# pi@R4:~/python/pkw/bin $ ./pkw_cli.py
# Traceback (most recent call last):
# File "./pkw_cli.py", line 14, in &module>
# import pkw_pkg
# File "/home/pi/python/pkw/bin/pkw_pkg/__init__.py", line 7, in &module>
# from . import pkw_initialize
# File "/home/pi/python/pkw/bin/pkw_pkg/pkw_initialize.py", line 6, in &module>
# import rrdtool
# ModuleNotFoundError: No module named 'rrdtool'
Here we find
pip install rrdtool
Note: unless binary versions are available for your target system,
the command above requires rrdtool development files (headers, libraries, dependencies) to be installed.
See python-rrdtool
python details
script header
A
shebang line defines where the interpreter is located
SheBang is ignored if we use this form :
$ python ./my_script.py
The general form ...
#!/usr/bin/env python
... executes whatever python is on the users $PATH.
This form is resilient to the python interpreter being moved around, which makes it somewhat more portable,
but it also means that the user can override the standard python interpreter by putting something ahead of it in $PATH.
Depending on your goals, this behavior may or may not be OK.
The specific form ...
#!/usr/bin/python
... deals with the common case that a python interpreter is installed in /usr/bin.
If it's installed somewhere else, you lose.
But this is a good way to ensure you get exactly the version you want or else nothing at all ("fail-stop" behavior).
stackoverflow
The most specific form ...
#!/usr/bin/python3
... forces to work with version 3, installed on a specific path {mychoice}
You can use
#!/usr/bin/env python3
# -*- coding: UTF-8 -*-
...for portability across different systems in case they have the language interpreter installed in different locations.
decoding UTF
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
encoding declarations
display python version runtime
nicolau@mars:~/sebas/python/veure_versions$ cat mostra_versio_python.py
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
import platform #
docs
import sys
print( "1 python version", platform.python_version() )
print( "2 python version", sys.version )
print( "3 python version", sys.version_info )
sys.exit()
Ens dona :
nicolau@mars:~/sebas/python/veure_versions$ ./mostra_versio_python.py
1 python version 3.7.5
2 python version 3.7.5 (default, Dec 9 2021, 17:04:37)
[GCC 8.4.0]
3 python version sys.version_info(major=3, minor=7, micro=5, releaselevel='final', serial=0)
Or maybe
if sys.version_info.major == 3:
print('*** python3')
else:
print('*** python2')
More complete :
pi@R4:~/python $ cat 1_veure_versio_python.py
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
# https://stackoverflow.com/questions/9079036/how-do-i-detect-the-python-version-at-runtime
import datetime # for timestamp
import sys
def check_installation(rv):
current_version = sys.version_info
print( "CV is {%i/%i}." % (current_version[0], current_version[1]) )
if current_version[0] == rv[0] and current_version[1] >= rv[1]:
pass
else:
sys.stderr.write( "[%s] - error: your python interpreter must be %d.%d or greater\n" % (sys.argv[0], rv[0], rv[1]) )
sys.exit(-1)
return 0
if __name__ == "__main__":
required_version = (2,7)
check_installation(required_version)
python parameters / arguments
Complete code with exceptions
Some simple parameters :
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
import sys
numArg = len(sys.argv)
print( 'R4 : /home/pi/eines/5_mostrar_parametres.py amb', numArg, 'parametres' )
idx = 0
for arg in sys.argv:
print( 'Idx (%i) : (%s).' % ( idx, arg ) )
idx = idx + 1
print( 'Number of arguments (%i).' % numArg )
print( 'Argument list (%s).' % sys.argv )
Num = 0
if ( numArg > 1 ) :
szAra = sys.argv[1]
if ( szAra == '?' ) :
print( 'provide date param in form "yyyy-mm-dd".' )
sys.exit()
myFilename = 'temperatura_' + szAra + '.txt'
sys.exit()
There are "non-keyword" parameters, and there are "keyword" parameters.
They have to be in that order, or you get a error message like this :
SyntaxError: non-keyword arg after keyword arg
selecting beVerbose or not from input argument
bVerbose = False
numArg = len(sys.argv)
if ( numArg > 1 ) :
szAra = sys.argv[1]
if ( szAra == '+' ) : bVerbose = True ;
if bVerbose: print('>>> comencem !')
function arguments and default values
python allows function arguments to have default values.
If the function is called without the argument, the argument gets its default value.
Default values indicate that the function argument will take that value if no argument value is passed during the function call.
The default value is assigned by using the assignment(=) operator of the form keywordname=value.
def student(firstname, lastname ='Mark', standard ='Fifth'):
some special operations
"/" vs "//"
/ is regular division (returns float) and // is floor division (returns int).
Floor division was introduced in python 3.
rjust()
right-justify and fill with char - w3schools :
string.rjust(length, character)
standard types that are built into the interpreter
docs.python :
- line.split() - split into parts
- line.strip() - remove leading ...
Single quote, double quote, triple quote, reverse quote
Generally, double quotes are used for string representation and single quotes are used for regular expressions, dict keys or SQL -
geeks 4 geeks
Double quotes around a string are used to specify a regular expression search - ks.uiuc
When a string contains special characters like quotes, we need to escape them :
>>> 'It\'s a good example.'
"It's a good example."
You can put a backslash character followed by a quote (\" or \').
This is called an escape sequence and python will remove the backslash, and put just the quote in the string.
We can use triple quotes (i.e., triplet of single quotes or triplet double quotes)
to represent the strings containing both single and double quotes to eliminate the need of escaping any :
>>> print('''She said, "Thank you! It's mine."''')
She said, "Thank you! It's mine."
A pending improvement :
template strings
console.log( '*** (' + genTimeStamp() + ') *** index DOM ready.' ) ;
console.log( `*** ( ${ genTimeStamp() } ) *** index DOM ready.` ) ;
var szTemp = genTimeStamp() + ' *** la temperatura a casa meva es ' + mijson.temp + ' ºC ***’ ;
var szTemp = `${ genTimeStamp() } *** la temperatura a casa meva es ${ mijson.temp } ºC ***` ;
Compte : removed backticks in python 3 !
curios problema de'n Ramon
Les cometes han de ser senzilles - si les posem dobles, tenim :
SERVER_HOST = "fronius"
--- value error : host can't be set (not a valid IP address or hostname)
python procedures and parameters
We can define some code to be used few times
#define a procedure
def procedureName(arg1, arg2, ...):
print('put instructions here')
#call the procedure
procedureName()
easy python docs
python procedures return values
A python function will always have a return value.
That default return value will always be None.
real python
Dealing with environment variables
To display all environment, do :
$ printenv
or
$ env
To read an environment variable from python, use os.environ :
import os
my_host = os.environ["HOSTNAME"] # can produce
library
Compte : "HOSTNAME" is NOT an envir var !
pubs.opengroup
Get it from "socket" :
import socket
szHost = socket.gethostname()
How to set a global envir var
Define it in "/etc/environment" and reboot :
sebas@pi0alby:~ $ cat /etc/environment
SAG_tg_tkn="1234567890987654321"
sebas@pi0alby:~/ $ echo $SAG_tg_tkn
1234567890987654321
Cant "export" it :
sebas@pi0alby:~ $ cat /etc/environment
export SAG_tg_tkn="1234567890987654321"
/vat/log/syslog : Sep 16 01:11:32 pi0alby systemd[527]: /usr/lib/environment.d/99-environment.conf:2:
invalid variable name "export SAG_tg_tkn", ignoring.
Verify it is visible :
pi@pi0:~/python/bot $ python3
Python 3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.environ['MYUSR']
'myusr'
>>> os.environ['MYPWD']
'mypwd'
>>> os.environ['MYTOKEN']
'1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghij'
How to read a global envir var into python
Use "os" :
sebas@pi0alby:~/python/envir $ cat 6_llegir_token.py
#!/usr/bin/env python
import os
import sys
my_token = os.environ["SAG_tg_tkn"]
lng = len( my_token )
print( 'llegit token [', my_token, '].' )
print( 'llargada [', lng, '].' )
sys.exit()
Aware of errors, "None" versus "KeyError" :
my_bot_token = os.environ['SAG_PI0_TOKEN'] # if non-existing, you get 'KeyError' exception
my_bot_token = os.environ.get('SAG_PI0_TOKEN') # using get will return 'None' if a key is not present rather than raise a 'KeyError'
try:
my_bot_token = os.environ["SAG_PI0_TOKEN"]
except Exception as e:
print("{0}".format(e.__class__))
read a global envir var, with default value if non-existing
DEVICE_ID = "35680858b4e62d74d557" # endoll ID
DEVICE_IP = "192.168.1.100" # endoll IP
DEVICE_KEY = "uxyew1YTvqqx8Yre" # endoll KEY - see "python3 -m tinytuya wizard"
DEVICE_VERS = "3.3"
# check for environmental variables and always use those if available
PLUGID = os.getenv('PLUGID', DEVICE_ID)
PLUGIP = os.getenv('PLUGIP', DEVICE_IP)
PLUGKEY = os.getenv('PLUGKEY', DEVICE_KEY)
PLUGVERS = os.getenv('PLUGVERS', DEVICE_VERS) #
OS = miscellaneous operating system interfaces
Lets improve the code with a "try" sentence :
try:
myIP = os.environ["SAG_ENDOLL_SP21_IP"]
print( '>>> envir IP is {}'.format(myIP) )
PLUGIP = os.getenv('SAG_ENDOLL_SP21_IP', DEVICEIP)
print( '>>> default IP is {}'.format(DEVICEIP) )
print( '>>> IP in use is {}'.format(PLUGIP) )
except KeyError as e:
print( '--- got a KeyError - reason {%s}' % str(e) )
print( '--- undefined SAG_ENDOLL_SP21_IP' )
except AttributeError as e:
print( "--- Attribute Error (%s) " % ( e ) )
sys.exit()
How to read envir var into python from a service
From a service, /etc/environment is no used.
So, we use EnvironmentFile [***]
How to run python code as a service
How to log from run python code when running as a service
from systemd import journal
journal.write("Hello Lennart")
ModuleNotFoundError: No module named 'systemd'
python3 -m pip install systemd
Display your log lines using journalctl
sebas@r4:~/python/telegram/telepot $ sudo journalctl --lines 0 --follow _SYSTEMD_UNIT=r4_bot.service
sys.exc_info
sys.exc_info docu
Un molt bon article de
linuxjournal.com/article/5821 : sys.exc_info() function
try:
...
except SomeException:
tb = sys.exc_info()[2]
raise OtherException(...).with_traceback(tb)
Un altre StackOverflow :
import sys
import traceback
def formatExceptionInfo(maxTBlevel=5):
cla, exc, trbk = sys.exc_info()
excName = cla.__name__
try:
excArgs = exc.__dict__["args"]
except KeyError:
excArgs = "<no args>"
excTb = traceback.format_tb(trbk, maxTBlevel)
return (excName, excArgs, excTb)
try:
x = x + 1
except:
print formatExceptionInfo()
Code from linuxjournal.com/article/5821 !!!
Handling exceptions
Exceptions are used to handle errors and other exceptional conditions that may occur during program execution.
Catching Any Exception:
try:
value = 10 / 0 # some code that might raise an exception
except Exception as e:
print(f"An error occurred: {e}")
See exception hierarchy and
KeyError
wiki :
try:
print("MY_HOME:", os.environ['MY_HOME'])
except KeyError as ke:
print( '--- got a KeyError {%s}' % str(ke) )
except AttributeError as ae:
print( "--- got a Attribute Error {%s}" % ae )
except NameError as ne:
print( "--- Name error: ", ne )
except:
print( "--- unexpected error: ", str( sys.exc_info()[0] ) )
A more complete example is :
try:
while True:
some_code
except ( KeyboardInterrupt, SystemExit ) :
exit_code_here
Nice code :
import os # import the os module
path = '/var/www'
try:
os.chdir(path)
print ( "+++ Current working directory: {0}".format(os.getcwd() ) )
except FileNotFoundError:
print("--- Directory: {0} does not exist".format(path))
except NotADirectoryError:
print("--- {0} is not a directory".format(path))
except PermissionError:
print("--- You do not have permissions to change to {0}".format(path))
except OSError as err : #
https://docs.python.org/3/tutorial/errors.html
print(err)
print("--- OS error: {0}".format(err))
linuxize
A bit more could be :
try:
doSomeEvilThing()
except Exception, e:
handleException(e)
raise # "raise" without passing an exception object causes the original traceback to be preserved
# typically it is much better than "raise e"
Complete one :
nicolau@mars:~/sebas/ramon/minicom_test$ cat 0_test.py
#!/usr/bin/env python
import sys
try:
x = x + 1
except:
print( sys.exc_info() )
nicolau@mars:~/sebas/ramon/minicom_test$ ./0_test.py
(<class 'NameError'>, NameError("name 'x' is not defined",), <traceback object at 0x7f66b016bb48>)
GPT says "basic syntax of exception handling"
try:
# Code that might raise an exception
except SomeException as e:
# Code that runs if an exception is raised
print(f"An error occurred: {e}")
else:
# Code that runs if no exception occurs
print("No errors occurred!")
finally:
# Code that always runs, regardless of whether an exception occurred or not
print("This will always run.")
A larger one :
try:
file2 = urllib.request.Request('site goes here')
file2.add_header("User-Agent", 'Opera/9.61 (Windows NT 5.1; U; en) Presto/2.1.1')
ResponseData = urllib.request.urlopen(file2).read().decode("utf8", 'ignore')
except socket.error as e: ResponseData = ''
except socket.timeout as e: ResponseData = ''
except UnicodeEncodeError as e: ResponseData = ''
except http.client.BadStatusLine as e: ResponseData = ''
except http.client.IncompleteRead as e: ResponseData = ''
except urllib.error.HTTPError as e: ResponseData = ''
except urllib.error.URLError as e: ResponseData = e.reason
be careful with catching the subclass of errors before their superclass.
In your example, this would mean putting HTTPError before URLError.
Otherwise, the subclass will never get caught.
The last except clause may omit the exception name(s), to serve as a wildcard :
try:
except:
print( "--- Unexpected error: ", str( sys.exc_info()[0] ) )
print( "--- Unexpected error: ", sys.exc_info() )
--- Unexpected error: (<class 'NameError'>, NameError("name 'psutil' is not defined",), <traceback object at 0x7f0f5aaf9b48>)
errors
Another source :
- in the event of a network problem (e.g. DNS failure, refused connection, etc), Requests will raise a ConnectionError exception.
- in the event of the rare invalid HTTP response, Requests will raise an HTTPError exception.
- if a request times out, a Timeout exception is raised.
- if a request exceeds the configured number of maximum redirections, a TooManyRedirects exception is raised.
- all exceptions that Requests explicitly raises inherit from requests.exceptions.RequestException.
You can either catch the base-class exception, which will handle all cases :
try:
r = requests.get(url, params={'s': thing})
except requests.exceptions.RequestException as e: # this is the correct syntax -
systemOverflow
raise SystemExit(e)
Or you can catch them separately and do different things :
try:
r = requests.get(url, params={'s': thing})
except requests.exceptions.Timeout:
# maybe set up for a retry, or continue in a retry loop
except requests.exceptions.TooManyRedirects:
# tell the user their URL was bad and try a different one
except requests.exceptions.RequestException as e:
raise SystemExit(e) # catastrophic error - bail.
read the docs - requests errors and exceptions
No puc treure els detalls de un "Name Error" :
docs.python ,
airbrake
Tampoc de un "TypeError" - code
except Exception as e:
print( 'Exception ' + str(e) ) # "e" te tipus "TypeError"
exemple complert errors requests.get
nicolau@mars:~/sebas/python/llegir_preu_electre/agafen_altres_dades_de_REE$ cat 10073_eolica.py
import json
try:
response = requests.get(url, headers=headers)
response.raise_for_status() # we want http errors (e.g. 401 Unauthorized) to raise exceptions
if response.status_code == 200:
json_data = json.loads(response.text) # parse a JSON string and convert into a dictionary
json_string = json.dumps(json_data, indent=4) # convert to a string
print ( '+++ JSON [%s].' % json_string )
else:
print( "--- status not 200 error: ", response.status_code )
# go from specific to general down the stack of errors
except requests.exceptions.HTTPError as errh: #
python 3 only
print ("--- Http Error:", errh)
raise SystemExit(errh)
except requests.exceptions.ConnectionError as errc:
print ("--- Error Connecting:", errc)
except requests.exceptions.Timeout as errt:
print ("--- Timeout Error:", errt)
except requests.exceptions.TooManyRedirects: # tell the user their URL was bad and try a different one
print ( '--- redirect.')
except requests.exceptions.RequestException as errex: #
python 3 only
raise SystemExit(errex)
SIGINT, Keyboard interrupt
import time, sys
x = 1
while True:
try:
print( x )
time.sleep(.3) # wait() a little bit
x += 1
except KeyboardInterrupt:
print( "--- kbd Ctrl-C : bye" )
sys.exit()
stacloverflow
TypeError exception
try:
except TypeError as te:
print(f"--- a TYPE error occurred: {te}")
ValueError exception
try:
except ValueError as ve:
print( "--- Value Error : ", ve )
NameError exception
In my_main() we can get this error message :
--- Unexpected error: <class 'NameError'>
Then, we have to code :
except NameError as ne:
print( "--- Name error: ", ne ) # output expected NameErrors
And now we can see the real cause of the error :
--- Name error: name 'psutil' is not defined
Resum : faltava "import psutil"
ALL exceptions
"complete hierarchy" from here from
here
BaseException
+-- SystemExit
+-- KeyboardInterrupt
+-- GeneratorExit
+-- Exception
+-- StopIteration
+-- StopAsyncIteration
+-- ArithmeticError
| +-- FloatingPointError
| +-- OverflowError
| +-- ZeroDivisionError
+-- AssertionError
+-- AttributeError
+-- BufferError
+-- EOFError
+-- ImportError
+-- ModuleNotFoundError
+-- LookupError
| +-- IndexError
| +-- KeyError
+-- MemoryError
+-- NameError
| +-- UnboundLocalError
+-- OSError
| +-- BlockingIOError
| +-- ChildProcessError
| +-- ConnectionError
| | +-- BrokenPipeError
| | +-- ConnectionAbortedError
| | +-- ConnectionRefusedError
| | +-- ConnectionResetError
| +-- FileExistsError
| +-- FileNotFoundError
| +-- InterruptedError
| +-- IsADirectoryError
| +-- NotADirectoryError
| +-- PermissionError
| +-- ProcessLookupError
| +-- TimeoutError
+-- ReferenceError
+-- RuntimeError
| +-- NotImplementedError
| +-- RecursionError
+-- SyntaxError
| +-- IndentationError
| +-- TabError
+-- SystemError
+-- TypeError
+-- ValueError
| +-- UnicodeError
| +-- UnicodeDecodeError
| +-- UnicodeEncodeError
| +-- UnicodeTranslateError
+-- Warning
+-- DeprecationWarning
+-- PendingDeprecationWarning
+-- RuntimeWarning
+-- SyntaxWarning
+-- UserWarning
+-- FutureWarning
+-- ImportWarning
+-- UnicodeWarning
+-- BytesWarning
+-- ResourceWarning
var scope and global vars
Python has no command for declaring a variable. A variable is created the moment you first assign a value to it.
nicolau@mars:~/sebas/python/proves$ cat global_var.py
#!/usr/bin/env python3
import sys
Idx = 0
def my_func() :
global Idx # (***)
Idx = 6
return Idx
RC = my_func()
print( "{%i-%i}" % (Idx, RC) )
Result is {0-6} or {6-6} depending on the presence of the {***} sentence
logging
sebas@pi0alby:~/python/telegram $ cat client.py
#!/usr/bin/env python3
import logging
import sys
import requests # python -m pip install requests
fnLog = '/home/sebas/logs/send_telegram.log'
szTxt = " *** R0 envia al bot del TG *** "
logging.basicConfig( filename=fnLog, level=logging.DEBUG, \
format='%(asctime)s %(message)s', datefmt='%d/%m/%Y %H:%M:%S' )
logging.info( '\n\n *** Anem a enviar al bot.' )
logging.warning( 'Texte {%s}', szTxt )
tg_rc = telegram_bot_sendtext( szTxt )
logging.debug( f'RC {tg_rc}' )
sys.exit()
logging basics
logging to a file
own log or bitacora
En lloc de fer "print()", cridem al nostre codi per afegir marca temporal :
def trassa( szTxt ):
now = datetime.datetime.now()
szNow = now.strftime("%Y-%m-%d %H:%M:%S ")
print( szNow + szTxt )
atexit()
atexit -
this only works with functions that can be called without arguments.
sebas@pi0alby:~/python $ cat mostrar_temperatura.py
#!/usr/bin/env python
import atexit # exit hook
import datetime # timestamp
import sys # to "print"
def all_done():
print( 'all_done()' )
print '>>> Registering'
atexit.register( all_done )
print '+++ Registered'
now = datetime.datetime.now()
szTime = now.strftime("%Y-%m-%d %H:%M:%S %Z")
temp = int(open('/sys/class/thermal/thermal_zone0/temp').read()) / 1e3
print( '[' + szTime + '] temp.CPU => ' + str(temp).' )
print 'Exiting...'
sys.exit()
Non-Ascii characters
Poso una "ç" en un comentari i surt :
File "./bot_pihole.py", line 35
SyntaxError: Non-ASCII character '\xc3' in file ./bot_pihole.py on line 35, but no encoding declared;
see http://python.org/dev/peps/pep-0263/ for details
Llegim http://python.org/dev/peps/pep-0263/
"if ... then ... else" statement
The complete form is :
if b > a:
print("b is greater than a")
< more statements >
else: # indentation defines the begin and end of the "then" and "else" clauses
print("b is not greater than a")
< other statements >
A short form in one line is :
if len(szH) == 1: szH = '0' + szH
I use a lot :
bVerbose = False # True
if bVerbose: print('>>> llegir la temperatura')
You can put few statements in one line :
if (j == 8): szOut += '\n' ; j = 0 ;
In python, you can have if-else statements on one line, using the conditional expression syntax:
some_expression if condition else other_expression
codingem
A compound if statement is written like this :
if (a > b) and (c < d):
nested "if" statements
if iVal == 0:
print("+++ select an operation")
elif iVal == 1:
print("+++ display switch status")
elif iVal == 2:
print("+++ set switch ON")
else:
print("--- invalid selection {}".format(iVal) )
python while loop
A simple loop is :
i = 1
while i < 6:
print(i)
i += 1
A "forever" loop is like this :
sebas@pi0alby:~/python/i2c $ cat MCP23008_flash.py
#!/usr/bin/python3
# MCP23008 address, 0x20(32)
# registre 9 = gpio data register
import smbus
import time
bus = smbus.SMBus(1) # set I2C bus
bus.write_byte_data(0x20, 0x00, 0x00) # all pins are configured as outputs
while True:
bus.write_byte_data(0x20, 0x09, 0x00)
time.sleep(0.5) # wait(), delay next operation
bus.write_byte_data(0x20, 0x09, 0xFF)
time.sleep(0.5) # delay
"print()" statement, python v3
In python 3, print() is a function
Syntax is (few samples) :
print 'tenim ' + str(nummsg) + ' msg' # straight
print 'Msg ID is (%s/%s).' % (num, nummsg) # 2 params
print( 'The value of PI is approx %5.3f.' % math.pi ) #
format
print( 'Disk total (%d), free (%d).' % ( my_disk_total, my_disk_free ) ) # 2 parameters
szName = "My name is {}, I'm {}".format( "John", 36 ) # el millor metode
txt1 = "My name is {fname}, I'm {age}".format(fname = "John", age = 36) # named indexes
agent_name = 'James Bond'
kill_count = 9
# old ways
print("%s has killed %d enemies" % (agent_name,kill_count))
print('{} has killed {} enemies'.format(agent_name,kill_count))
print('{name} has killed {kill} enemies'.format(name=agent_name,kill=kill_count))
# f-strings way
print(f'{agent_name} has killed {kill_count} enemies')
Mind that the "print(" initial parentheses has to be closed at the very end of line, after the "% (params)"
format specifiers
Lots of print format tricks :
'{1} {0}'.format('one', 'two') -> 'two one'
The %s specifier converts the object using str(), and %r converts it using repr() -
url
how to format integers, strings, etc
hv, lv = tuple( struct.pack('>H', v ) ) # split an integer (2 bytes) into 2
# "<" is Little Endian, ">" is Big Endian
chH = f'{ hv:02X }' # convert integer into hex, specifying length
"as" sentence
El as es per fer un alias, i que no hagis de canviar totes les referencies en cas que canviis la llibreria.
Molt util per saltar de versions.
import sebastia as sebas
sebas.desperta()
sebas.festa_major()
Pero de cop ara ets el sebastianot oi?
import sebastianot as sebas
Compte : "old" name is not valid anymore, so we can find this error :
C:\python\> type nmpy.py
import numpy as np
print "NumPy version %s" % numpy.__version__
C:\python> python nmpy.py
Traceback (most recent call last):
File "nmpy.py", line 3, in <module>
print "NumPy version %s" % numpy.__version__
NameError: name 'numpy' is not defined
python "bytes" object
A "bytes" object represents a sequence of bytes, a "string" object represents a sequence of unicode code points.
To convert between from "bytes" to "string" and from "string" back to "bytes"
you use the bytes.encode and string.decode functions.
These functions take two parameters, an encoding and an error handling policy.
python circular list
"itertools" or
this
conn = ['a', 'b', 'c', 'd', 'e', 'f']
conn_len = len(conn)
index = 0
while True:
print(conn[index])
index = (index + 1) % conn_len
python collection data types
There are four collection data types in the python programming language:
- list is a collection which is ordered and changeable. Allows duplicate members.
Square brackets : thislist = [ "apple", "banana", "cherry" ]
You can use the list as an array.
- tuple is a collection which is ordered and unchangeable. Allows duplicate members.
Round brackets : thistuple = ( "apple", "banana", "cherry" )
- set is a collection which is unordered and unindexed. No duplicate members.
Curly brackets : thisset = { "apple", "banana", "cherry" }
- dictionary is a collection which is unordered, changeable and indexed. No duplicate members.
Have keys and values.
Curly brackets : thisdict = { "brand": "Ford", "model": "Mustang", "year": 1964 }
w3 schools
Use type() function to print the data type of a var :
data = d.status()
if (d):
print( type( data['dps']['1'] ) ) # output is ... <class 'bool'>
bVal = data['dps']['1'] #
In python programming, key-value pairs are dictionary objects and ordered list are list objects.
Brief refresher on the techniques used to extract data from a JSON structure.
Data Type Example Extraction Method Python Code
---------- -------- ------------------ ------------
List animal = [cat, dog. horse. bird] Index operator with square brackets [] print(animal[0]) # output: 'cat'
Dictionary student = {'name': Joe, 'age': 26} Keys with square brackets [] print(student['name']) # output: 'Joe'
python lists
A list is a collection which is ordered and changeable. In python lists are written with square brackets.
Each item in the list has a value (color name) and an index (its position in the list). Python uses zero-based indexing.
thislist = [ "apple", "banana", "cherry" ]
print(thislist)
length of a list
inp_lst = [ 'Python', 'Java', 'Kotlin', 'Machine Learning', 'Keras' ]
size = len(inp_lst)
print(size)
looping through a list
Print all items in the list :
thislist = [ "apple", "banana", "cherry" ]
for x in thislist:
print(x)
Print all items by referring to their index number:
thislist = [ "apple", "banana", "cherry" ]
for i in range( len( thislist )):
print( thislist[i] )
Print all items, using a while loop to go through all the index numbers
thislist = [ "apple", "banana", "cherry" ]
i = 0
while i < len( thislist ):
print( thislist[i] )
i = i + 1
python tuples
A tuple is a collection which is ordered and unchangeable. In python tuples are written with round brackets.
thistuple = ( "apple", "banana", "cherry" )
print(thistuple)
A real sample :
sebas@pi0alby:~/python/tinet $ ./get_internal_ip.py
IPs of network adapter wlan0
ip type [<class 'str'>]
ip0 [192.168.1.222]
pfx 24
python sets
A set is a collection which is unordered and unindexed. In python sets are written with curly brackets.
thisset = { "apple", "banana", "cherry" }
print(thisset)
The reason the set iteration order changes from run-to-run appears to be because Python uses hash seed randomization by default.
stackOverflow
python set length
# creating a set
myset = {"USA", "CANADA", "BRAZIL"}
# implementing the len() function
print(len(myset))
educative.io
iterate over a python set
fruits = { "Apple", "Peach", "Mango" }
for fruit in fruits:
print(fruit)
programiz
python dictionaries
A dictionary is a collection which is unordered, changeable and indexed.
In python dictionaries are written with curly brackets, and they have keys and values.
thisdict = { "brand": "Ford", "model": "Mustang", "year": 1964 }
print(thisdict)
objChat = { 'first_name': 'Sebastia',
'id': 304588090,
'last_name': 'Altemir',
'type': 'private' }
print ("Original dictionary is : " + str(objChat)) # print the dictionary
an overview of keys and values in dictionaries in python
Keys inside a python dictionary can only be of a type that is immutable.
Immutable data types in python are integers, strings, tuples, floating point numbers, and Booleans.
Dictionary keys cannot be of a type that is mutable, such as sets, lists, or dictionaries.
When it comes to values inside a python dictionary there are no restrictions.
Another thing to note about the differences between keys and values in python dictionaries, is the fact that keys are unique.
This means that a key can only appear once in the dictionary, whereas there can be duplicate values.
freecodecamp
create an empty dictionary and add elements to it
my_dict = {} # create an empty dictionary ...
my_dict = dict() # another way : use the dict() function without passing any arguments
my_dict[my_key] = my_val # afegim al nou diccionari
change values in a dictionary
You can change the value of a specific item by referring to its key name:
thisdict = {
"brand": "Ford",
"model": "Mustang",
"year": 1964
}
thisdict["year"] = 2018
looping through a dictionary
You can loop through a dictionary by using a for loop.
When looping through a dictionary, the return value are the keys of the dictionary, but there are methods to return the values as well.
Some details : url
objChat = { 'first_name': 'Sebastia',
'id': 304588090,
'last_name': 'Altemir',
'type': 'private' }
print ("Dict key-value are : ")
for i in objChat : # using in operator to get key and value
print(i, objChat[i])
Dict key-value are :
id 304588090
first_name Sebastia
last_name Altemir
type private
accessing a specific dictionary element
objChat = { 'first_name': 'Sebastia',
'id': 304588090,
'last_name': 'Altemir',
'type': 'private' }
myId = objChat['id']
myUserName = objChat['last_name']
myFirstName = objChat['first_name']
delete a dictionary entry : "del()" or "pop()" ?
geeks
delete a specific dictionary element
Have to create a "shadow" copy not to get "RuntimeError: dictionary changed size during iteration" :
my_dict = { 'a': 1, 'b': 2, 'c': 3 }
for key in my_dict.copy():
print(key)
if key == 'b':
del my_dict[key]
print(my_dict) # 👉️ {'a': 1, 'c': 3}
bobby hadz
sort
Sorting a python dictionary: values, keys and more
file modes in file handling
r Open text file for reading
r+ Open for reading and writing
w Create text file for writing
w+ Open for reading and writing
a Open for writing/append
The file is created if it does not exist
The stream is positioned at the end of the file
a+ Open for reading and writing
The file is created if it does not exist
The stream is positioned at the end of the file
+++
r open for reading. The file pointer is placed at the beginning of the file. This is the default mode.
r+ Opens a file for both reading and writing. The file pointer will be at the beginning of the file.
w Opens a file for writing only. Overwrites the file if the file exists. If the file does not exist, creates a new file for writing.
w+ Opens a file for both writing and reading. Overwrites the existing file if the file exists.
If the file does not exist, it creates a new file for reading and writing.
rb Opens a file for reading only in binary format. The file pointer is placed at the beginning of the file.
rb+ Opens a file for both reading and writing in binary format.
wb+ Opens a file for both writing and reading in binary format. Overwrites the existing file if the file exists.
If the file does not exist, it creates a new file for reading and writing.
a Opens a file for appending. The file pointer is at the end of the file if the file exists.
If the file does not exist, it creates a new file for writing.
ab Opens a file for appending in binary format. The file pointer is at the end of the file if the file exists.
If the file does not exist, it creates a new file for writing.
a+ Opens a file for both appending and reading. The file pointer is at the end of the file if the file exists.
If the file does not exist, it creates a new file for reading and writing.
ab+ Opens a file for both appending and reading in binary format. The file pointer is at the end of the file if the file exists.
If the file does not exist, it creates a new file for reading and writing.
x open for exclusive creation, failing if the file already exists (Python 3)
Read / write files
write to a file
Simple way :
with open( szFQFN, "a+" ) as fOut:
fOut.write( szOut ) # .close() called automatically
Complete :
fd = open( "demofile2.txt", "a" ) # append mode
fd.write("Now the file has more content!")
fd.close()
read from a file
fd = open('myfile.txt', 'r')
Elegant :
try:
reader = open('dog_breeds.txt')
# further file processing goes here
finally:
reader.close()
Another :
try:
f = open(arg, 'r')
except OSError:
print('--- cannot open', arg)
else:
print(arg, 'has', len(f.readlines()), 'lines')
f.close()
Working with :
CSV,
JSON
llegim un fitxer linia a linia
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
iFN = 'in.txt'
fd = open( iFN, 'r' )
count = 0
while True:
line = fd.readline() # get next line from file
if not line: # if line is empty
break # .. then end of file is reached -> break the loop
# .. else
count += 1
print( "Line {}: [{}].".format( count, line.strip() ) )
fd.close()
convertir CSV a diccionari
MY_SAMPLE.CSV
a,1
b,2
d = dict()
f = open("my_sample.csv")
for line in f:
line = line.strip() # remove trailing \n
(key, val) = line.split(",") # split the line
d[key] = val
print(d)
OUTPUT
{'a': '1', 'b': '2'}
adam smith.haus
how to parse a string
parse a string
szNumMsg = str(nummsg) # string is ['80']
newMsgNum = szNumMsg.split("'")[1] # got plain "80"
(key, PP, PC, PD) = line.split(",") # split CSV into 3 vars
See split CSV
use "partition"
If it's always going to be an even LHS/RHS split,
you can also use the partition method that's built into strings.
It returns a 3-tuple as (LHS, separator, RHS) if the separator is found, and (original_string, '', '') if the separator wasn't present:
>>> "2.7.0_bf4fda703454".partition('_')
('2.7.0', '_', 'bf4fda703454')
and/or
>>> "shazam".partition("_")
('shazam', '', '')
STACKOVERFLOW
Another sample :
el@apollo:~/foo$ python
>>> mystring = "theres coffee in that nebula"
>>> mytuple = mystring.partition(" ")
>>> print type(mytuple)
<type 'tuple'>
>>> print mytuple
('theres', ' ', 'coffee in that nebula')
>>> print mytuple[0]
theres
>>> print mytuple[2]
coffee in that nebula
convert TXT/LOG to HTML
header = '<!doctyle html><html><head><title>My Title</title></head><body>'
body = '<table><thead><tr><th>Col 1</th><th>Col 2</th></tr>'
footer = '</table></body></html>'
with open('input.txt', 'r') as input, open('output.html', 'w') as output:
output.writeln(header)
output.writeln(body)
for line in input:
col1, col2 = line.rstrip().split()
output.write('<tr><td>{}</td><td>{}</td></tr>\n'.format(col1, col2))
output.write(footer)
stackOverflow
python debug
Use python -m pdb <program name> to go step-by-step thru your program.
Complete documentation
Basic debug commands :
- use "n" to execute actual line and stop at next line - skip subroutine
- use "s" to execute actual line and stop at first possible occasion - go into subroutine
- use "q" to quit program.
C:\sebas\miscosas\albert\python\webcam> python -m pdb 3-v-en-color.py
> c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(1)()
-> import numpy as np
(Pdb) next
> c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(2)()
-> import cv2
(Pdb) next
> c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(6)()
-> cap = cv2.VideoCapture(0)
(Pdb) next
> c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(8)()
-> while(True):
(Pdb) next
> c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(11)()
-> ret, frame = cap.read()
(Pdb) next
> c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(17)()
-> if ret:
(Pdb) next
> c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(18)()
-> print dir ( frame ) ;
(Pdb) next
['T', '__abs__', '__add__', '__and__', '__array__', '__array_finalize__', '__array_interface__', '__array_prepare__', '_
_array_priority__', '__array_struct__', '__array_wrap__', '__class__', '__contains__', '__copy__', '__deepcopy__', '__de
lattr__', '__delitem__', '__delslice__', '__div__', '__divmod__', '__doc__', '__eq__', '__float__', '__floordiv__', '__f
ormat__', '__ge__', '__getattribute__', '__getitem__', '__getslice__', '__gt__', '__hash__', '__hex__', '__iadd__', '__i
and__', '__idiv__', '__ifloordiv__', '__ilshift__', '__imod__', '__imul__', '__index__', '__init__', '__int__', '__inver
t__', '__ior__', '__ipow__', '__irshift__', '__isub__', '__iter__', '__itruediv__', '__ixor__', '__le__', '__len__', '__
long__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__', '__neg__', '__new__', '__nonzero__', '__oct__', '__or__
', '__pos__', '__pow__', '__radd__', '__rand__', '__rdiv__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '
__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '
__rtruediv__', '__rxor__', '__setattr__', '__setitem__', '__setslice__', '__setstate__', '__sizeof__', '__str__', '__sub
__', '__subclasshook__', '__truediv__', '__xor__', 'all', 'any', 'argmax', 'argmin', 'argpartition', 'argsort', 'astype'
, 'base', 'byteswap', 'choose', 'clip', 'compress', 'conj', 'conjugate', 'copy', 'ctypes', 'cumprod', 'cumsum', 'data',
'diagonal', 'dot', 'dtype', 'dump', 'dumps', 'fill', 'flags', 'flat', 'flatten', 'getfield', 'imag', 'item', 'itemset',
'itemsize', 'max', 'mean', 'min', 'nbytes', 'ndim', 'newbyteorder', 'nonzero', 'partition', 'prod', 'ptp', 'put', 'ravel
', 'real', 'repeat', 'reshape', 'resize', 'round', 'searchsorted', 'setfield', 'setflags', 'shape', 'size', 'sort', 'squ
eeze', 'std', 'strides', 'sum', 'swapaxes', 'take', 'tobytes', 'tofile', 'tolist', 'tostring', 'trace', 'transpose', 'va
r', 'view']
> c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(19)()
-> cv2.imshow('3.py - frame',frame)
(Pdb)
python breakpoint()
Insert "breakpoint()" to enter debug -
realpython
python trace statement execution
The trace module can be invoked from the command line. It can be as simple as
python -m trace --count -C . somefile.py ...
doc.python.org {amb programmatic interface},
pymotw.com
python debug links
Here's a
list
of resources to get started with the python debugger:
print dir
Display built-in functions, methods, modules and module attributes.
$ python -v
>>> import ModbusClient
Traceback (most recent call last):
ModuleNotFoundError: No module named 'ModbusClient'
>>> from pyModbusTCP.client import ModbusClient
>>> print(dir(ModbusClient))
encoding
Problema 20210227 :
sebas@pi0alby:~/python/tinet $ python -m pdb ./enviar_html_a_tinet.py
Traceback (most recent call last):
File "/usr/lib/python2.7/pdb.py", line 1314, in main
pdb._runscript(mainpyfile)
File "/usr/lib/python2.7/pdb.py", line 1233, in _runscript
self.run(statement)
File "/usr/lib/python2.7/bdb.py", line 400, in run
exec cmd in globals, locals
File "<string>", line 1, in <module>
File "./enviar_html_a_tinet.py", line 57
SyntaxError: Non-ASCII character '\xc3' in file ./enviar_html_a_tinet.py on line 57, but no encoding declared;
see http://python.org/dev/peps/pep-0263/ for details
Solucio - posar com a segona linia el seguent :
# -*- coding: UTF-8 -*-
user-defined functions
A function is a block of organized, reusable code that is used to perform a single, related action.
Functions provide better modularity for your application and a high degree of code reusing.
All parameters (arguments) in the python language are passed by reference.
tutorials point ,
code mentor
See r/w JSON sample
default parameters
Often when defining a function, there are certain values that we want the function to use most of the time, but we'd also like to give the user some flexibility.
In this case, we can use default values for arguments.
The values can also be specified by name if desired, in which case the order of the named values does not matter.
lambda functions
Another way of defining short, one-off functions
add = lambda x, y: x + y
add(1, 2)
This lambda function is roughly equivalent to
def add(x, y):
return x + y
So why would you ever want to use such a thing? Primarily, it comes down to the fact that everything is an object in Python, even functions themselves!
That means that functions can be passed as arguments to functions.
WhirlWindTour ch.08
Calling system functions
We shall use subprocess :
#!/usr/bin/env python
import subprocess
import os
import sys
# res = subprocess.Popen(['ls','-al','/ahome'],stdout=subprocess.PIPE,stderr=subprocess.PIPE);
# output,error = res.communicate()
# if res.returncode:
# #raise Exception(error)
# print "error>>>> ",res.returncode
# else:
# print "output>>>> ",output
try:
res = subprocess.Popen(['ls','-al','/home'],stdout=subprocess.PIPE,stderr=subprocess.PIPE);
output,error = res.communicate()
if output:
print "ret> ", res.returncode
print "OK> output ", output
if error:
print "ret> ", res.returncode
print "Error> error ", error.strip()
#except CalledProcessError as e:
# print "CalledError > ",e.returncode
# print "CalledError > ",e.output
except OSError as e:
print "OSError > ", e.errno
print "OSError > ", e.strerror
print "OSError > ", e.filename
except:
print( "Error > " + str( sys.exc_info()[0] ) )
gist
Another sample :
mate@punt-omnia:~/eines/python/server_status$ cat sesta.py
#!/usr/bin/python
# https://docs.python.org/2/contents.html
# https://docs.python.org/2/library/subprocess.html
import subprocess
fname="/home/mate/eines/python/server_status/testfile.txt"
mfile = open( fname,"w" )
mfile.write( "+++ (1) Date is\n" )
mcmd = 'date'
print 'My comand #1 is [%s].' % mcmd
out_date = subprocess.check_output( mcmd )
mfile.write( out_date )
mfile.write( "+++ (2) Temp is\n" )
mcmd = '/usr/bin/sensors'
print 'My comand #2 is [%s].' % mcmd
out_sensors = subprocess.check_output( mcmd )
mfile.write( out_sensors )
mfile.write( "+++ (3) Fin\n" )
mfile.close()
calling a script with arguments
import subprocess
val = subprocess.check_call("./script.sh '%s'" % arg, shell=True)
Remember to convert the args to string before passing, using str(arg).
This can be used to pass as many arguments as desired:
subprocess.check_call("./script.ksh %s %s %s" % (arg1, str(arg2), arg3), shell=True)
I recommend using list as arguments like
subprocess.check_call(["./script.ksh", arg1, arg2, arg3], shell=True)
StackOverflow
my code : calling a script with arguments
import subprocess
path_to_local_dir = '/home/pi/python/pkw/eines/analitzar_un_dia'
nomShell = 'extreure_1_dia.sh'
fnShell = path_to_local_dir + '/' + nomShell
szCMD = fnShell + " -d " + szFN + " -i " + "%s" % eINI + " -f " + "%s" % eFIN + " -o " + "%s" % fn_dades_1_dia
print ( "CMD ***{}".format (szCMD) )
iRC = subprocess.call( [szCMD], shell=True )
print("Extreure return code", iRC)
System monitoring
Shell scripts for system monitoring from
here
cmd = "hostname -I | cut -d' ' -f1"
IP = subprocess.check_output(cmd, shell=True).decode("utf-8")
cmd = "top -bn1 | grep load | awk '{printf \"CPU Load: %.2f\", $(NF-2)}'"
CPU = subprocess.check_output(cmd, shell=True).decode("utf-8")
cmd = "free -m | awk 'NR==2{printf \"Mem: %s/%s MB %.2f%%\", $3,$2,$3*100/$2 }'"
MemUsage = subprocess.check_output(cmd, shell=True).decode("utf-8")
cmd = 'df -h | awk \'$NF=="/"{printf "Disk: %d/%d GB %s", $3,$2,$5}\''
Disk = subprocess.check_output(cmd, shell=True).decode("utf-8")
datetime details
Read datetime docu
strftime
The strftime() method returns a string representing date and time using date, time or datetime object.
now = datetime.datetime.now() # getdatetime object
szInici = now.strftime("%Y-%m-%d %H:%M:%S %Z") # convert to string
strptime
The strptime() method creates a datetime object from the given string.
do_Time = datetime.strptime( szTime, "%Y-%m-%dT%H:%M:%S.%f") # datetime object
We can split it into separated values :
import datetime
now = datetime.datetime.now()
print(now.year, now.month, now.day, now.hour, now.minute, now.second)
And improve it a bit :
year = '{:02d}'.format(now.year)
month = '{:02d}'.format(now.month)
day = '{:02d}'.format(now.day)
hour = '{:02d}'.format(now.hour)
minute = '{:02d}'.format(now.minute)
Generate a timestamp
In python, dates are objects.
Therefore, when you manipulate them, you manipulate objects, not strings, not timestamps nor anything.
Any object in python have TWO string representations:
- the regular representation that is used by "print", can be get using the str() function.
It is most of the time the most common human readable format and is used to ease display.
So str(datetime.datetime(2008, 11, 22, 19, 53, 42)) gives you '2008-11-22 19:53:42'.
- the alternative representation that is used to represent the object nature (as a data).
It can be get using the repr() function
and is handy to know what kind of data your manipulating while you are developing or debugging.
So repr(datetime.datetime(2008, 11, 22, 19, 53, 42)) gives you 'datetime.datetime(2008, 11, 22, 19, 53, 42)'.
ya@punt-omnia:~/eines/monit_campanar$ cat monit_campanar.py
import datetime
now = datetime.datetime.now()
szInici = now.strftime("%Y-%m-%d %H:%M:%S %Z")
saltycrane
Write daily measures into same file and change automagically
pi@R4:~/python/pkw/eines/veure_potencia_instant $ cat 4_veure_potencia_instant_i_esciure_a_fitxer.py
szPath_Soroll = '/home/pi/python/soroll/csv'
now = datetime.datetime.now()
szTemps = now.strftime("%Y-%m-%d %H:%M:%S") # used in filename
szHora = now.strftime("%H:%M") # used in CSV
szFN_soroll_CSV = "soroll_CSV_%s.csv" % ( now.strftime("%Y_%m_%d") ) # dont use HH ni MM ni SS in Filename - the file contains a full DAY
szFQFN_soroll_CSV = szPath_Soroll + '/' + szFN_soroll_CSV
szOutCSV = szHora + ',' + str( int(Soroll) ) + '\n' # create a CSV line
if beVerbose:
print( ">>> (%s) Open (%s)." % (szTemps, szFQFN_soroll_CSV) )
fSorollCSV = open( szFQFN_soroll_CSV, "a+") # open to Append
try:
fSorollCSV.write( szOutCSV ) # write CSV
if beVerbose:
print( "+++ (%s) Write (%s)." % (szTemps, szOutCSV) )
finally:
fSorollCSV.close() # close file
if beVerbose:
print( "+++ (%s) Close." % (szTemps) )
Mesurem gap
inici = datetime.datetime.now()
szDades = urllib2.urlopen( my_url ).read().decode(encoding='UTF-8').strip() # agafem dades del Fronius
final = datetime.datetime.now()
delta = final - inici # get difference
print( "Time difference is {%s} seconds" % delta.total_seconds() ) # time difference in seconds
ms = delta.total_seconds() * 1000
print( "Time difference is {%s} milliseconds" % ms ) # time difference in milliseconds
pynative
Calling python code from bash
You execute a python code from bash like this
python python_script.py arg1 arg2
And to access these arguments in python you can use:
import sys
print sys.argv[0] # prints python_script.py
print sys.argv[1] # prints arg1
print sys.argv[2] # prints arg2
The python results have to be available to bash :
result=$(python3 python_code.py)
Calling python code from node
Lots of ways and details,
and python-shell module and API
Working sample - the python code ( after "npm install python-shell" )
app.get( '/fer_foto', function (req, res) {
// var PythonShell = require( 'python-shell' ) ; // send commands to python
let {PythonShell} = require( 'python-shell' ) ; // send commands to python since v 1.0.7
var python_options = { //
full documentation and source code, github
mode: 'text',
pythonPath: '/usr/bin/python',
pythonOptions: ['-u'],
scriptPath: '/home/pi/semafor',
args: ['value1', 'value2', 'value3']
} ;
console.log( '>>> Hacer foto.' );
PythonShell.run( 'fer_foto.py', python_options, function( err, results ) {
if ( err ) {
var szErr = '--- Python error. ' ;
szErr += 'Path (' + err.path + '). ' ;
szErr += 'Stack (' + err.stack + '). ' ;
console.log( szErr ) ;
throw err ; // fatal error : stop
} else {
console.log( '(+) Python results are (%j).', results ) ; // results is an array consisting of messages collected during execution
png_File = String( results[0] ) ; // convert to string
mConsole( "(+) python filename (" + png_File + ")." ) ;
res.writeHead( 200, { 'Content-Type': 'application/json' }) ;
let myjson = { status: 'OK', imgURL: png_File };
res.end( JSON.stringify( myjson ) ) ;
} ; // no error
} ) ;
} ) ; // fer foto
And the result is :
>>> Hacer foto.
(+) Python results are (["hi","kukurutxu"]).
Taking pictures by python using a USB webcam
On ho faig servir (gracies, Albert) :
- //odin/home/sag/express-sendfile/server.js crida "fer_foto.py"
- //odin/home/pi/semafor/1_sem.js crida "2_foto.py"
- //r0/home/pi/njs/fronius/servidor.js crida "3_ramon.py"
Calling python code from python
The server.py shall be like this
$ cat server.py
def server_job(arg1, arg2, etc):
# do whatever the script does
retirun iRC
if __name__ == "__main__":
RC = server_job(sys.argv[1], sys.argv[2], sys.argv[3])
And we can call it like this :
$ cat client.py
import server
RC = server.server_job(params)
numpy = numerical python
To know the version we have :
import numpy
print "### NUMPY version I have is [%s]." % numpy.__version__
Lots of
uses
pandas
pandas is a Python package providing fast, flexible, and expressive data structures
designed to make working with “relational” or “labeled” data both easy and intuitive.
pandas
doc,
options
10 minutes intro
pandas options
pd.set_option('display.max_rows', None) # https://dev.to/chanduthedev/how-to-display-all-rows-from-data-frame-using-pandas-dha
pd.set_option('display.max_colwidth', None) # https://www.statology.org/pandas-column-width/
TCP/IP
Consider
Results,
an elegant and simple HTTP library for python, built for human beings.
TCP/IP :
- wireshark + la opcio "follow stream" per veure seguir el que es parlen
- python "requests" estirant del "payload" , o be alguna de les mil llibreries tcp/ip / arp,
depen de on es trobi el dialeg que vols "destripar":
TCP
pibcast-1.0.tar.gz
get MAC
pi@R4:~$ python3 -m pip install getmac
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting getmac
Downloading https://files.pythonhosted.org/packages/ea/ad/7821cbe5394fcb/getmac-0.8.3-py2.py3-none-any.whl
Installing collected packages: getmac
The script getmac is installed in '/home/pi/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed getmac-0.8.3
geeks -> es un script !
pi@R4:/usr/local/bin $ cat getmac
#!/usr/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from getmac.__main__ import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(main())
get host name, internal IP, external IP, ...
To get TCP/IP local info into python we use :
# https://pythonhosted.org/ifaddr/
import ifaddr # pip install ifaddr
import socket
szHost = socket.gethostname() # get "hostname"
adapters = ifaddr.get_adapters()
for adapter in adapters:
print( "IPs of network adapter " + adapter.nice_name )
for ip in adapter.ips:
print( " %s/%s" % (ip.ip, ip.network_prefix) )
my_URL = 'http://ipv4.icanhazip.com'
my_IP_ext = urlopen( my_URL ).read().decode(encoding='UTF-8').strip() # convert bytes to string and remove trailing \n
print( '+++ my ext IP [', my_IP_ext, ']' )
simple IP
There is a simple way to get the IP :
from subprocess import check_output
ip4 = check_output(['hostname', '-I']).decode(encoding='UTF-8').strip() # convert bytes to string and remove trailing \n
print( ">>> ip4 [%s]." % ip4 )
Explicacion en "man hostname" :
-I, --all-ip-addresses
Display all network addresses of the host.
This option enumerates all configured addresses on all network interfaces.
The loopback interface and IPv6 link-local addresses are omitted.
Contrary to option -i, this option does not depend on name resolution.
Do not make any assumptions about the order of the output.
complicated IP
sebas@pi0alby:~/python/i2c/oled $ cat /home/sebas/python/i2c/oled/b_continous_2ip.py
# This function allows us to grab any of our IP addresses
def get_ip_address( ifname ):
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
return socket.inet_ntoa(
fcntl.ioctl(
s.fileno(),
0x8915, # SIOCGIFADDR
struct.pack("256s", str.encode(ifname[:15])),
)[20:24]
)
ip_wifi = get_ip_address("wlan0")
ip_lan = get_ip_address("eth0")
get wifi name
nicolau@mars:~/sebas/python/ping_GAS2$ cat wifi_name.py
#!/usr/bin/env python3
import subprocess
import sys
subprocess_result = subprocess.Popen('iwgetid',shell=True,stdout=subprocess.PIPE)
subprocess_output = subprocess_result.communicate()[0],subprocess_result.returncode
network_name = subprocess_output[0].decode('utf-8').strip()
print( "name (%s)" % network_name )
sys.exit()
how to ping() from python
nicolau@mars:~/sebas/python/icmp$ cat 1_ping.py
#!/usr/bin/env python3
# -*- coding: UTF-8 -*-
from icmplib import ping # https://github.com/ValentinBELYN/icmplib
myIP = '192.168.1.1'
host = ping( myIP, count=4, interval=1, timeout=2, privileged=True)
print( ">>> ping IP (%s)." % myIP )
if host.is_alive:
print(f'+++ {host.address} is alive! avg_rtt={host.avg_rtt} ms')
else:
print(f'--- {host.address} is dead')
El resultat es :
nicolau@mars:~/sebas/python/icmp$ sudo ./1_ping.py
>>> ping IP (192.168.1.1).
+++ 192.168.1.1 is alive! avg_rtt=3.952 ms
icmplib
icmplib at github
pyping
Use it like this :
mate@punt-omnia:~/nodejs-projects/timer$ cat 2_do_ping.py
#!/usr/bin/python
# we have to return 2 strings :
# 1) the IP we are ping-ing
# 2) the result
import sys
import pyping
# pip install pyping
# https://github.com/certator/pyping/blob/master/bin/pyping
numArg = len( sys.argv )
idx = 0
szDesti = sys.argv[1] # in nodejs we code "python_options.args[0] = szIP ;"
print 'IP (',szDesti,').' # ret string #1
try:
response = pyping.ping( szDesti, timeout=800, count=3 ) # -t 900 -c 2
if response.ret_code == 0:
print( "+++ RC 0 - Viagra, reachable" ) # 74.125.143.104
else:
print( "--- RC KO - Promescent, unreachable" ) # 1.2.3.4
except ZeroDivisionError:
print( "=== RC KO - divBy0" )
how to ftp() from python
STORLINES vs
STORBINARY
opening URLs
Lets compare "requests" to "urlopen"
my_url = 'http://ifconfig.me/ip'
response = requests.get(my_url)
print( '0 my IP : {}'.format( response.text.strip() ) )
my_IP = urlopen( 'http://icanhazip.com' ).read() # get a "bytes" object
print( '1 my IP :', my_IP )
my_IP = urlopen( 'http://icanhazip.com' ).read().decode(encoding='UTF-8') # ... convert "bytes" object to string
print( '2 my IP :', my_IP )
my_IP = urlopen( 'http://icanhazip.com' ).read().decode(encoding='UTF-8').strip() # ... and remove trailing \n
print( '3 my IP :', my_IP )
my_IP = requests.get( "https://api.ipify.org/?format=json" ).json()['ip'] # site provides JSON format
print( '4 my IP :', my_IP )
A complete JSON info, as geolocation
homepage , including how to get external IP using 21 languages, as Lua, nim, jQuery, Scala ...
requests
requests is requests : HTTP for Humans -
see docu advanced section
Minimum code :
import requests # python -m pip install requests
sitestring_input = input("Enter a domain (ie, www.google.com): ")
my_url = "http://%s" % [sitestring_input] # getting the URL string formatted safely
try:
req = requests.get(my_url)
except:
print("nope, sorry")
Have to add some code to manage
errors and exceptions or
timeouts
urllib
See code at TG.HTM
import urllib.request
import urllib.error
import json
while True:
my_url = input('Enter a url like:(www.google.com)-> ')
try:
resp = urllib.request.urlopen( my_url, timeout = 5000 )
json_data = json.loads( resp )
except urllib.error.HTTPError as err_h:
print(err_h.reason)
break # break the loop - otherwise "sys.exit()"
except urllib.error.URLError as err_u:
print(err_u.reason)
break
else:
print('URL Exists')
break
reddit
compte urllib.error sota python 2 o python 3
pi@R4:~$ python
Python 2.7.16 (default, Oct 10 2019, 22:02:15)
[GCC 8.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.error
Traceback (most recent call last):
File "&stdin>", line 1, in &module>
ImportError: No module named error
>>>
pi@R4:~/python/pkw/bin/pkw_pkg $ python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.error funciona OK sota python 3
>>>
display module contents - urllib
>>> print(dir(urllib))
['ContentTooShortError', 'FancyURLopener', 'MAXFTPCACHE', 'URLopener', '__all__', '__builtins__', '__doc__', '__file__',
'__name__', '__package__', '__version__', '_asciire', '_ftperrors', '_have_ssl', '_hexdig', '_hextochr', '_hostprog',
'_is_unicode', '_localhost', '_noheaders', '_nportprog', '_passwdprog', '_portprog', '_queryprog',
'_safe_map', '_safe_quoters', '_tagprog', '_thishost', '_typeprog', '_urlopener', '_userprog', '_valueprog',
'addbase', 'addclosehook', 'addinfo', 'addinfourl', 'always_safe', 'base64', 'basejoin', 'c',
'ftpcache', 'ftperrors', 'ftpwrapper', 'getproxies', 'getproxies_environment', 'i', 'localhost', 'noheaders',
'os', 'pathname2url', 'proxy_bypass', 'proxy_bypass_environment', 'quote', 'quote_plus', 're', 'reporthook',
'socket', 'splitattr', 'splithost', 'splitnport', 'splitpasswd', 'splitport', 'splitquery', 'splittag',
'splittype', 'splituser', 'splitvalue', 'ssl', 'string', 'sys', 'test1', 'thishost', 'time', 'toBytes',
'unquote', 'unquote_plus', 'unwrap', 'url2pathname', 'urlcleanup', 'urlencode', 'urlopen', 'urlretrieve']
urllib at Raspberry versus Ubuntu
The same code has to be changed a bit, and I still dont know why : {p2 versus p3 ?}
urllib at Raspberry
pi@R4:~/python/fronius $ cat llegir_energia.py
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
# agafem dades del Fronius
import json
import urllib
hostname = "fronius" # defined in /etc/hosts
def main(argv):
endpoint = "/solar_api/v1/GetPowerFlowRealtimeData.fcgi"
my_url = "http://" + hostname + endpoint
szDades = urllib.urlopen( my_url ).read().decode(encoding='UTF-8').strip()
objjson = json.loads( szDades ) # parse a JSON string and convert into a dictionary
energia_del_dia = objjson["Body"]["Data"]["Inverters"]["1"]["E_Day"]
now = datetime.datetime.now()
szTime = now.strftime("%Y-%m-%d %H:%M:%S %Z")
print( "+++ {%s} todays energy = (%d) {Wh}." % (szTime, energia_del_dia ) )
if __name__ == "__main__":
main(sys.argv)
urllib at Ubuntu
nicolau@mars:~/sebas/python/fronius$ cat 8_llegir_energia.py
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
import json
# import urllib # raspberry
import urllib.request # --- 1st change
hostname = "fronius" # defined in /etc/hosts
def main(argv):
endpoint = "/solar_api/v1/GetPowerFlowRealtimeData.fcgi"
my_url = "http://" + hostname + endpoint
# szDades = urllib.urlopen( my_url ).read().decode(encoding='UTF-8').strip() # raspberry
szDades = urllib.request.urlopen( my_url ).read().decode(encoding='UTF-8').strip() # --- 2nd change
objjson = json.loads( szDades ) # parse a JSON string and convert into a dictionary
energia_del_dia = objjson["Body"]["Data"]["Inverters"]["1"]["E_Day"]
now = datetime.datetime.now()
szTime = now.strftime("%Y-%m-%d %H:%M:%S %Z")
print( "+++ {%s} todays energy = (%d) {Wh}." % (szTime, energia_del_dia ) )
if __name__ == "__main__":
main(sys.argv)
urllib2
urllib2 — extensible library for opening URLs - better known exceptions !
urllib2 provides some extra functionality, namely the urlopen() function can allow you to specify headers
docs.python.org :
v 2,
v 3
import urllib2 # raspbian - https://docs.python.org/2/library/urllib2.html
import json
endpoint = "/solar_api/v1/GetMeterRealtimeData.cgi?Scope=System"
my_url = "http://" + hostname + endpoint
print( "\n>>> (2) Read (%s)." % endpoint )
try:
szDades = urllib2.urlopen( my_url, timeout = 5 ).read().decode(encoding='UTF-8').strip()
objjson = json.loads( szDades ) # parse a JSON string and convert into a dictionary
p_power_real = objjson["Body"]["Data"]["0"]["PowerReal_P_Sum"]
print( "+++ real power [GetMeterRealtimeData PowerReal_P_Sum] - (%d) {W}. As P_Grid." % p_power_real )
except urllib2.HTTPError as e:
print('HTTP error = ' + str(e.code) )
except urllib2.URLError as e:
print('URL error = ' + str(e.reason) )
except KeyboardInterrupt:
print( "INTERRUPTED!" )
no funciona amb python3
The urllib2 module has been split across several modules in python 3 named urllib.request and urllib.error -
python 3.8 ,
urllib
my_url = urllib2.Request('http://www.example.com', postBackData, { 'User-Agent' : 'My User Agent' })
try:
response = urllib2.urlopen( my_url, timeout = 5 )
except urllib2.HTTPError as e:
checksLogger.error('HTTPError = ' + str(e.code))
except urllib2.URLError as e:
checksLogger.error('URLError = ' + str(e.reason))
except httplib.HTTPException as e:
checksLogger.error('HTTPException')
except Exception:
import traceback
checksLogger.error('generic exception: ' + traceback.format_exc())
stack overrun
urllib2 is the name of the library included in Python 2. You can use the urllib.request library included with Python 3, instead.
url
Alternatively, install the requests library for a higher-level and easier to use API. It'll work on both Python 2 and 3
python requests
Requests: HTTP for Humans
Homepage,
API docu
requests will work on both Python 2 and 3
Quickstart,
advanced documentation
resp = requests.get('https://api.github.com/events')
resp = requests.post('https://httpbin.org/post', data={'key': 'value'})
resp = requests.put('https://httpbin.org/put', data={'key': 'value'})
resp = requests.delete('https://httpbin.org/delete')
resp = requests.head('https://httpbin.org/get')
resp = requests.options('https://httpbin.org/get')
Here is a complete sample :
import requests
import json
try:
response = requests.get( my_url )
response.raise_for_status() # we want HTTP errors (as 401 Unauthorized) to raise exceptions
if response.status_code == 200:
json_data = json.loads( response.text ) # parse a JSON string and convert into a dictionary
json_string = json.dumps(json_data, indent=4) # convert to a string
print ( '+++ (1) JSON [%s].' % json_string )
# ... or ...
jsondata = response.json() # use builtin JSON decoder -
requests
jsonstring = json.dumps( jsondata, indent=4, sort_keys=True ) # convert a python dictionary into a JSON string
print ( '+++ (2) JSON [%s].' % jsonstring ) # *** (1) and (2) are identical ***
else:
print( "--- status not 200 error: ", response.status_code )
except requests.exceptions.HTTPError as errh: #
python 3 only
print( "--- HTTP error: ", errh )
except requests.exceptions.ConnectionError as errc:
print( "--- error connecting: ", errc )
except requests.exceptions.Timeout as errt:
print( "--- timeout error: ", errt )
except requests.exceptions.TooManyRedirects: # tell the user their URL was bad and try a different one
print( '--- redirect.' )
except requests.exceptions.RequestException as errex: #
python 3 only
print( '--- request exception:', errex )
raise SystemExit(errex)
Errors and exceptions
In the event of a network problem (e.g. DNS failure, refused connection, etc), Requests will raise a ConnectionError exception
Response.raise_for_status() will raise an HTTPError if the HTTP request returned an unsuccessful status code
If a request times out, a Timeout exception is raised
If a request exceeds the configured number of maximum redirections, a TooManyRedirects exception is raised
All exceptions that Requests explicitly raises inherit from requests.exceptions.RequestException
CURL requests to pvoutput
Parameters to the GET request, as "h=1" : {url} : "URL&h=1"
The GET request expects sid and key
while the POST expects X-Pvoutput-SystemId and X-Pvoutput-ApiKey.
forum
To set HTTP headres, use "-H" : curl -H "X-Header: value" https://www.keycdn.com
$ curl -H "X-Pvoutput-Apikey: MYKEY" -H "X-Pvoutput-SystemId: MYID" 'https://pvoutput.org/service/r2/getstatus.jsp?&h=1&limit=288&d=20221027'
Otherwise, use sid and key keywords :
$ curl 'https://pvoutput.org/service/r2/getstatus.jsp?sid=MYSID&key=MYKEY&h=1&limit=288&d=20221028'
Date Time Ener Efici Pow Ave Normalised
gy ency er rage
| | | | | | |
20221028,18:15,15796,2.480,120,120,0.019,NaN,NaN,NaN,NaN;
20221028,18:10,15786,2.478,163,168,0.026,NaN,NaN,NaN,NaN;
20221028,18:05,15772,2.476,193,192,0.030,NaN,NaN,NaN,NaN;
20221028,18:00,15756,2.473,296,300,0.047,NaN,NaN,NaN,NaN;
20221028,17:55,15731,2.470,385,384,0.060,NaN,NaN,NaN,NaN;
20221028,17:50,15699,2.465,499,504,0.079,NaN,NaN,NaN,NaN;
20221028,17:45,15657,2.458,556,552,0.087,NaN,NaN,NaN,NaN;
20221028,17:40,15611,2.451,551,552,0.087,NaN,NaN,NaN,NaN;
20221028,17:35,15565,2.443,564,564,0.089,NaN,NaN,NaN,NaN;
20221028,17:30,15518,2.436,642,648,0.102,NaN,NaN,NaN,NaN;
20221028,17:25,15464,2.428,727,732,0.115,NaN,NaN,NaN,NaN;
20221028,17:20,15403,2.418,662,660,0.104,NaN,NaN,NaN,NaN;
20221028,17:15,15348,2.409,670,672,0.105,NaN,NaN,NaN,NaN;
20221028,17:10,15292,2.401,714,720,0.113,NaN,NaN,NaN,NaN;
python requests to pvoutput
Download historical solar photovoltaic data from PVOutput.org :
python code
Managing RESTful API
REST stands for "representational state transfer" and is a software architecture style that defines a pattern for client and server communications over a network.
REST defines the following architectural constraints:
- stateless: The server won’t maintain any state between requests from the client.
- client-server: The client and server must be decoupled from each other, allowing each to develop independently.
- cacheable: The data retrieved from the server should be cacheable either by the client or by the server.
- uniform interface: The server will provide a uniform interface for accessing resources without defining their representation.
- layered system: The client may access the resources on the server indirectly through other layers such as a proxy or load balancer.
- code on demand (optional): The server may transfer code to the client that it can run, such as JavaScript for a single-page application.
realpython -> "Flask", "Django REST Framework", "FastAPI"
{Albert, 20241223} - bottlepy
HTTP methods
REST APIs listen for HTTP methods like GET, POST, and DELETE to know which operations to perform on the web service’s resources.
A resource is any data available in the web service that can be accessed and manipulated with HTTP requests to the REST API.
The HTTP method tells the API which action to perform on the resource.
CRUD operations to HTTP methods :
- GET retrieves resources
- POST submits new data to the server
- PUT updates existing data
- DELETE removes data
For example, here’s one of the URLs for GitHub’s REST API :
https://api.github.com/users/<username>
This URL allows you to access information about a specific GitHub user.
Un altre exemple :
$ curl -i http://127.0.0.1:5000/countries \
-X POST \
-H 'Content-Type: application/json' \
-d '{"name":"Germany", "capital": "Berlin", "area": 357022}'
HTTP/1.0 201 CREATED
Content-Type: application/json
...
{
"area": 357022,
"capital": "Berlin",
"id": 4,
"name": "Germany"
}
This curl command has some options that are helpful to know :
- -i include the HTTP response headers in the output
- -X sets the HTTP method for the request
- -H adds an HTTP header to the request
- -d defines the request data
Mind Best practices for REST API security: Authentication and authorization
Un servidor de requeriments REST em sembla que ens empeny directament a
nodeJS i Express
REST API server using python
simplest python socket example, server side:
import socket
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.bind(('localhost', 8089))
serversocket.listen(5) # become a server socket, maximum 5 connections
while True:
connection, address = serversocket.accept()
buf = connection.recv(64)
if len(buf) > 0:
print buf
break
Or you can use HTTPServer module alongwith SimpleHTTPRequestHandler to create a simple webserver that serves your GET and POST request
python REST API server by Gemini
Input : "write a minimal REST server in python, request is pure text "
This version of the server is designed to receive and process plain text data sent in the body of a POST request.
Output :
sebas@minie:~/dades/python/rest$ cat rest_server.py
#!/usr/bin/python3
# -*- coding: utf-8 -*-
# install Flask using Mint Software Manager -> python3-flask
from flask import Flask, request, Response
import logging
import traceback
app = Flask(__name__)
# Configure logging
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
@app.route('/upload', methods=['POST'])
def upload_text():
"""
Handles text uploads. The text data is expected in the body of the POST request.
"""
if request.method == 'POST':
# Check if the content type is correct. We expect text/plain
if not request.headers['Content-Type'].startswith('text/plain'):
logging.error(f"--- Invalid content type: {request.headers['Content-Type']}. Expected text/plain")
return Response("Invalid content type. Expected text/plain", status=400)
# Check if there is data in the request.
if not request.data:
logging.error("--- Empty request body")
return Response("Empty request body", status=400)
try:
# Read the text data from the request as a string.
text_data = request.data.decode('utf-8') # Decode the byte data to a string
logging.info(f"+++ Received text: {text_data}")
# Process the text data as needed. Here, we just log it.
# You might save it to a file, database, or perform some other action.
# Generate a response
response_text = f"+++ Text received and processed successfully: {text_data}"
return Response(response_text, status=200)
except Exception as e:
# Log the error with the full traceback.
logging.error(f"---Error processing text: {e}\n{traceback.format_exc()}")
# Return a 500 Internal Server Error
return Response(f"Error processing text: {e}", status=500)
else:
# If the method is not POST
return Response("--- used method is not POST, not allowed", status=405)
if __name__ == '__main__':
# Start the Flask development server. For production, use a proper WSGI server.
app.run(debug=True, host='0.0.0.0', port=5000)
Details :
Content Type Handling:
The server checks for Content-Type: text/plain.
The error message reflects the expected content type.
Data Handling:
The code reads the request data as bytes using request.data and then decodes it into a string using .decode('utf-8').
The example processing logs the received text. You can replace this with your actual text processing logic.
Response:
The server sends a text response, including the received data.
Lets run it :
sebas@minie:~/dades/python/rest$ ./rest_server.py
* Serving Flask app 'rest_server' (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: on
2025-05-13 17:00:03,688 - WARNING - * Running on all addresses.
WARNING: This is a development server. Do not use it in a production deployment.
2025-05-13 17:00:03,688 - INFO - * Running on http://192.168.1.147:5000/ (Press CTRL+C to quit)
2025-05-13 17:00:03,690 - INFO - * Restarting with stat
2025-05-13 17:00:03,785 - WARNING - * Debugger is active!
2025-05-13 17:00:03,838 - INFO - * Debugger PIN: 631-935-736
The client is written in bsh and can be found
here
REST API server using C
crec recordar que era ... socket() + bind() + listen() + recv()
C REST API server URL's
- JeffreytheCoder : Simple-HTTP-Server -
dev.to &
github
When a client connects, the server accepts the connection and creates a new thread to handle the client's HTTP request.
This way, the server can handle multiple clients concurrently.
- simple web server with C programming , uses "fork()"
REST API docs and URLs
using VPN fom code
Si no vols que tota la màquina passi per la VPN, sinó només el teu programa, pots utilitzar un Proxy SOCKS5.
Molts serveis de VPN ofereixen aquesta opció.
Pots fer servir una eina anomenada proxychains.
- instal·lació: sudo apt install proxychains
- configuració: edita /etc/proxychains.conf per afegir la IP i el port del proxy de la teva VPN.
- ús: executes el teu script així: proxychains python3 el_teu_script.py
on trobar VPN servers
Estatics :
IP Address P ort País Velocitat Estimada
184.174.68.21 4145 Estats Units Mitjana
98.162.25.7 4145 Estats Units Baixa
72.223.168.67 4145 Estats Units Alta
192.252.209.155 14455 Estats Units Mitjana
On trobar-ne més en temps real?
- Spys.one: probablement la base de dades més completa. Pots filtrar per SOCKS5 i per país.
- Proxy-List.download: molt senzill per obtenir llistes en format text.
- Free-Proxy-List.net: ideal per a proxys ràpids, tot i que se centra més en HTTP/HTTPS.
Com provar-los des de la teva terminal d'Ubuntu?
Abans de posar-lo al teu programa Python, pots verificar si el proxy està viu amb curl:
curl --socks5-hostname 184.174.68.21:4145 https://ifconfig.me
Si et retorna una IP diferent de la teva, el proxy funciona.
proxy local : Tor
Si el teu objectiu és fer scraping de moltes URLs, els proxys gratuïts et donaran molts errors de connexió (ConnectionTimeout).
Per a un projecte seriós a Ubuntu 18.04, et sortiria molt més a compte utilitzar Tor com a proxy local:
- sudo apt install tor
- el teu proxy SOCKS5 local serà 127.0.0.1:9050.
- és gratuït, anònim i molt més estable que els proxys públics.
Tor - configuracio
- instal·lem el dimoni de Tor (el motor que corre en segon pla):
sudo apt update
sudo apt install tor -y
- Configuració del Proxy (fitxer /etc/tor/torrc [servidor] -> torsocks.conf [client])
SocksPort 9050 # port a fer servir TorPort
SocksListenAddress 127.0.0.1 # TorAddress
MaxCircuitDirtiness 300 # canvi de circuit cada 300 seg = 5 min
Log notice file /var/log/tor/notices.log
- Reiniciar i verificar
sudo systemctl restart tor # Reinicia el servei
netstat -an | grep 9050 # Comprova si el port 9050 està escoltant
- Per estar segur que Tor funciona abans de llançar el programa massiu, executa això a la terminal:
curl --socks5 127.0.0.1:9050 https://check.torproject.org/api/ip
Si et respon amb un JSON que diu {"IsTor":true, ...}, la teva configuració és perfecta.
- Com utilitzar-lo amb el teu programa
A. Sense tocar el codi (Recomanat):
Fes servir l'utilitat torsocks. Aquesta eina intercepta les crides de xarxa del teu script i les desvia cap al proxy local de Tor automàticament.
torsocks python3 el_teu_script.py
B. Especificant el proxy manualment:
Si el teu programa permet configurar un proxy, les dades que has d'introduir són:
Protocol: socks5
IP: 127.0.0.1
Port: 9050
Tor - depuracio
- sudo -u debian-tor tor --verify-config
- sudo -u debian-tor tor
- sudo systemctl restart tor
- sudo systemctl stop tor
- sudo killall -9 tor # Per si hi ha processos zombis
- sudo systemctl start tor
- netstat -an | grep 9050
- sudo ss -tulpn | grep tor - ha de sortir el port 9050
- sudo tail -n 50 /var/log/tor/log
Missatge curios :
At least one protocol listed as required in the consensus is not supported by this version of Tor.
You should upgrade.
This version of Tor will not work as a client on the Tor network.
The missing protocols are: FlowXtrl=1
Tor - actualitzacio
Netegem :
sudo apt-get purge tor
sudo apt-get autoremove
Engeguem :
sebastia@t60:~/Downloads/tor-browser_en-US/Browser/TorBrowser/Tor$ ./tor --SocksPort 9050 --DataDirectory ~/.tor_data
Feb 17 16:35:55.098 [notice] Tor 0.4.5.10 (git-fd74f7628eba2525) running on Linux with Libevent 2.1.8-stable, OpenSSL 1.1.1, Zlib 1.2.11, Liblzma N/A, Libzstd N/A and Glibc 2.27 as libc.
Feb 17 16:35:55.098 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Feb 17 16:35:55.099 [notice] Configuration file "/var/tmp/dist/tor/etc/tor/torrc" not present, using reasonable defaults.
Feb 17 16:35:55.104 [notice] Opening Socks listener on 127.0.0.1:9050
Feb 17 16:35:55.104 [notice] Opened Socks listener connection (ready) on 127.0.0.1:9050
Feb 17 16:35:55.000 [notice] Bootstrapped 0% (starting): Starting
Feb 17 16:35:55.000 [notice] Starting with guard context "default"
Feb 17 16:35:56.000 [notice] Bootstrapped 5% (conn): Connecting to a relay
Feb 17 16:36:02.000 [notice] Bootstrapped 10% (conn_done): Connected to a relay
Feb 17 16:36:06.000 [notice] Bootstrapped 14% (handshake): Handshaking with a relay
Feb 17 16:36:06.000 [notice] Bootstrapped 15% (handshake_done): Handshake with a relay done
Feb 17 16:36:06.000 [notice] Bootstrapped 20% (onehop_create): Establishing an encrypted directory connection
Feb 17 16:36:06.000 [notice] Bootstrapped 25% (requesting_status): Asking for networkstatus consensus
Feb 17 16:36:06.000 [notice] Bootstrapped 30% (loading_status): Loading networkstatus consensus
Feb 17 16:36:10.000 [notice] I learned some more directory information, but not enough to build a circuit: We have no usable consensus.
Feb 17 16:36:10.000 [notice] Bootstrapped 40% (loading_keys): Loading authority key certs
Feb 17 16:36:13.000 [warn] At least one protocol listed as recommended in the consensus is not supported by this version of Tor.
You should upgrade.
This version of Tor will eventually stop working as a client on the Tor network.
The missing protocols are: FlowCtrl=2 Relay=4
Feb 17 16:36:13.000 [notice] The current consensus has no exit nodes. Tor can only build internal paths, such as paths to onion services.
Feb 17 16:36:13.000 [notice] Bootstrapped 45% (requesting_descriptors): Asking for relay descriptors
Feb 17 16:36:13.000 [notice] I learned some more directory information, but not enough to build a circuit:
We need more microdescriptors: we have 0/9785, and can only build 0% of likely paths.
We have 0% of guards bw, 0% of midpoint bw, and 0% of end bw (no exits in consensus, using mid) = 0% of path bw.
Feb 17 16:36:13.000 [notice] Bootstrapped 50% (loading_descriptors): Loading relay descriptors
Feb 17 16:36:18.000 [notice] The current consensus contains exit nodes. Tor can build exit and internal paths.
Feb 17 16:36:30.000 [notice] Bootstrapped 56% (loading_descriptors): Loading relay descriptors
Feb 17 16:36:32.000 [notice] Bootstrapped 63% (loading_descriptors): Loading relay descriptors
Feb 17 16:36:33.000 [notice] Bootstrapped 69% (loading_descriptors): Loading relay descriptors
Feb 17 16:36:35.000 [notice] Bootstrapped 75% (enough_dirinfo): Loaded enough directory info to build circuits
Feb 17 16:36:35.000 [notice] Bootstrapped 80% (ap_conn): Connecting to a relay to build circuits
Feb 17 16:36:35.000 [notice] Bootstrapped 85% (ap_conn_done): Connected to a relay to build circuits
Feb 17 16:36:35.000 [notice] Bootstrapped 89% (ap_handshake): Finishing handshake with a relay to build circuits
Feb 17 16:36:35.000 [notice] Bootstrapped 90% (ap_handshake_done): Handshake finished with a relay to build circuits
Feb 17 16:36:35.000 [notice] Bootstrapped 95% (circuit_create): Establishing a Tor circuit
Feb 17 16:36:36.000 [notice] Bootstrapped 100% (done): Done
T60 with Ubuntu 18.04.4 is too old to run latest Tor.
Managing CSV
Sample :
pi@R4:~/python/pkw/eines/factura $ cat 2_csv_test.py
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
import sys
pfnCSV = 'csv_test.csv' # formated lines as "00:02,0,-316,316"
def leer_csv ( fnCSV ) :
n = 0
fdIN = open( fnCSV, 'r' )
for line in fdIN:
n = n + 1
line = line.strip() # remove trailing \n
(key, PP, PC, PD) = line.split(",") # split CSV using "," separator
print ( "CSV ({}) {} | {} | {} | {}".format (n, key, PP, PC, PD) )
fdIN.close()
if __name__ == "__main__":
leer_csv ( pfnCSV )
sys.exit()
Also pi@R4:~/python/pkw/eines/veure_potencia_instant/5_verificar_fitxer_csv.py and
pi@R4:~/python/pkw/eines/factura/1_factura.py
Input and output :
pi@R4:~/python/pkw/eines/factura $ more csv_test.csv
00:00,0,-322,322
00:01,0,-329,329
00:02,0,-316,316
00:03,0,-398,398
pi@R4:~/python/pkw/eines/factura $ ./2_csv_test.py
CSV (1) 00:00 | 0 | -322 | 322
CSV (2) 00:01 | 0 | -329 | 329
CSV (3) 00:02 | 0 | -316 | 316
CSV (4) 00:03 | 0 | -398 | 398
python read JSON data from CSV file
Use CSV module, specifically
class csv.DictReader ( f, fieldnames=None, restkey=None, restval=None, dialect='excel', *args, **kwds )
Create an object that operates like a regular reader
but maps the information in each row to a dict whose keys are given by the optional fieldnames parameter.
Trick : if fieldnames is omitted, the values in the first row of file f will be used as the fieldnames.
python examples - compte amb el format
els noms dels camps son a 1a linia si no es defineixen explicitament !
my_fieldnames = ['date', 'PP', 'PC', 'PD']
with open(csvFilePath, 'r') as csvf:
csvReader = csv.DictReader(csvf, fieldnames=my_fieldnames) # load csv file data into a dictionary r
python and JSON
JSON can store lists, bools, numbers, tuples and dictionaries.
But to be saved into a file, all these structures must be reduced to strings, so they can be read or written to a file.
- json_string = json.dumps ( my_dictionary )
Convert the python dictionary into a JSON string that can be written into a file.
- my_dictionary = json.loads ( json_string )
Convert a JSON string into a dictionary structure.
- json.load() : read JSON data from a file and convert it into a dictionary
- json.dump() : write python serialized object as JSON formatted data into a file
Sample :
import json
response = requests.get(url, timeout=60)
response.raise_for_status()
if response.status_code == 200:
json_data = json.loads(response.text) # parse a JSON string and convert into a dictionary
json_string = json.dumps(json_data, indent=4) # convert to a string
print ( '+++ (2) JSON [%s].' % json_string )
else:
print( "--- status not 200 error: ", response.status_code )
See complete doc
python json.load(s) sample
load() and loads() :
- to parse JSON from URL or file, use json.load()
- to parse string with JSON content, use json.loads()
import json
with open("developer.json", "r") as read_file: # read JSON file
developer = json.load(read_file) # convert JSON encoded data into python dictionary
for key, value in developer.items(): # display decoded JSON data from file
print(key, ":", value)
A mi m'agrada sense "with" :
import json
f = open('data.json') # open JSON file
dict = json.load(f) # get JSON object as a dictionary *** does NOT keep order in file ***
f.close() # close file
for key, value in dict.items(): # display JSON data
print(key, ":", value)
python json.dump(s) details
dump() and dumps() :
- json.dump() method is used to write Python serialized object as JSON formatted data into a file
- json.dumps() method encodes any Python object into JSON formatted string
Serialize obj to a JSON formatted string using this conversion table.
json.dumps( obj, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True,
cls=None, indent=None, separators=None, default=None, sort_keys=False, **kw )
python displays JSON data with nice format
import json
url = "http://" + hostname + endpoint
r = requests.get(url, timeout=60)
r.raise_for_status()
jsondata = r.json()
print( json.dumps( jsondata, indent=4, sort_keys=True ) )
python writes/reads JSON data to a file with nice format
hostname = "fronius"
version = "1.8.1-9"
def get_example(endpoint):
print("Fetching " + endpoint)
url = "http://" + hostname + endpoint
r = requests.get(url, timeout=60)
r.raise_for_status()
jsondata = r.json()
m = re.match(r".*/(.*)\..*", endpoint)
filename = m.group(1)
directory = os.path.join("examples", "json", version)
if not os.path.exists(directory):
os.makedirs(directory)
with open(os.path.join(directory, filename + ".json"), 'w') as f: # output file = GetAPIVersion.json
json.dump(jsondata, f, indent=4, sort_keys=True)
def main(argv):
get_example("/solar_api/GetAPIVersion.cgi")
if __name__ == "__main__":
main(sys.argv)
python read/write JSON data from file
The JSON data has this format :
{
"qq": 0,
"ts_ini": "2022-07-25 09:30:04",
"ts_fin": "2022-07-24 11:00:04"
}
And we manage it like this :
#!/usr/bin/env python3
import datetime
import json
import os
import sys
myFN = "jb_json.txt"
def My_Main():
try:
pf = open( myFN, 'r' )
mj = json.load( pf ) # read json from file into dictionary
print( mj )
mq = mj['qq']
mt = mj['ts']
print( "Q = ", mq )
print( "TS = ", mt )
pf.close()
pf = open( myFN, 'w' )
mj['qq'] = 0
mj['ts'] = "12:12:12, 02/03/2012"
json.dump( mj, pf ) # write json to file
pf.close()
except:
print( "--- Unexpected error: ", str( sys.exc_info()[0] ) )
if __name__ == '__main__':
My_Main()
sys.exit() # bye
python plot graph using JSON files
educative.io : How to plot graphs using .json files in Python
Easy "line" graph and easy "bar" graph
consells de'n Albert per fer una grafica amb python
- Vols programar? Mira python pyplot.
Es potser la millor virgueria que hi ha.
- No vols programar ni automatitzar?
Obre't un compte a databricks.
Li enxufes el json i ell solet s'ho empassa tot i ho transforma a una base de dades on pots fer de tot i mes, inclus grafics clar.
Funciona amb una cosa q es diu Jupiter.
Si no el coneixes t'el recomano per aixo i moltes altres coses, incluida la del punt 1 a dalt.
Es un python interactiu ficat dins del teu navegador, tot en local.
matplotlib
Que es ? plotting library
Matplotlib along with NumPy can be treated as the open source equivalent of MATLAB.
Tutorial,
examples,
nice guide,
documentacio 3.9.0
instalacio
matplotlib.org
Just as we use the np shorthand for NumPy and the pd shorthand for Pandas, we will use some standard shorthands for Matplotlib imports:
import matplotlib as mpl
import matplotlib.pyplot as plt
The plt interface is what we will use most often.
O'Reilly "Python Data Science Handbook" by Jake VanderPlas -
Chapter 4. Visualization with Matplotlib {****}
versio
$ python3 -c 'import matplotlib; print(matplotlib.__version__, matplotlib.__file__)'
import matplotlib
print( "Version ({})".format(matplotlib.__version__) )
I a les meves maquines :
sebas@minie:~/dades/_local_tinet_files$ python3 -c 'import matplotlib; print(matplotlib.__version__, matplotlib.__file__)'
3.5.1 /usr/lib/python3/dist-packages/matplotlib/__init__.py
pi@pi0:~/njs/fronius $ python3 -c 'import matplotlib; print(matplotlib.__version__, matplotlib.__file__)'
3.5.2 /home/pi/.local/lib/python3.7/site-packages/matplotlib/__init__.py
pi@pi0:~/njs/fronius $ python2 -c 'import matplotlib; print(matplotlib.__version__, matplotlib.__file__)'
Instalacio
Si en executar un python ens trobem amb
ModuleNotFoundError: No module named 'matplotlib'
... cal instalar "matplotlib"
Minie 20240520 - version 3.5.1-2build1
Pi0 :
pi@pi0:~/njs/fronius $ sudo apt-get install python-matplotlib
pi@pi0:~/njs/fronius $ python2 -c 'import matplotlib; print(matplotlib.__version__, matplotlib.__file__)'
('2.2.3', '/usr/lib/python2.7/dist-packages/matplotlib/__init__.pyc')
Conceptes : APIs i names of objects
Matplotlib is strongly object oriented and its principal objects are the figure and the axes.
You can think of the figure as a canvas, of which you typically specify the dimensions and possibly e.g., the background color etc etc.
You use the canvas, the figure, essentially in two ways, placing other objects on it (mostly axes, but also text labels etc) and saving its contents with savefig.
You can think of an axes as a sort of Swiss Army knife, a handy object that offers a tool (e.g. .plot, .scatter, .hist etc) for everything, mostly.
You can place one, two, ... many axes inside a figure using one of many different methods.
stackoverflow
Read
The Matplotlib Object Hierarchy
Matplotlib has two interfaces - API reference
- pyplot interface (function-based, implicit)
consists of functions in the pyplot module. Figure and Axes are manipulated through these functions and are only implicitly present in the background.
API: matplotlib.pyplot
- Axes interface (object-based, explicit)
create a Figure and one or more Axes objects, then explicitly use methods on these objects to add data, configure limits, set labels etc.
API: "subplots" - create Figure and Axes ; "axes" - add data, limits, labels etc. ; "Figure" - for figure-level methods
... an Axes actually translates into what we think of as an individual plot or graph (rather than the plural of “axis”)
Two interfaces for the price of one
A potentially confusing feature of Matplotlib is its dual interfaces:
a convenient MATLAB-style state-based interface,
and a more powerful object-oriented interface.
MATLAB-style interface
Matplotlib was originally written as a Python alternative for MATLAB users, and much of its syntax reflects that fact.
The MATLAB-style tools are contained in the pyplot (plt) interface.
It’s important to note that this interface is stateful: it keeps track of the “current” figure and axes, which are where all plt commands are applied.
You can get a reference to these using the plt.gcf() (get current figure) and plt.gca() (get current axes) routines.
plt.figure() # create a plot figure
# create the first of two panels and set current axis
plt.subplot(2, 1, 1) # (rows, columns, panel number)
plt.plot(x, np.sin(x))
# create the second panel and set current axis
plt.subplot(2, 1, 2)
plt.plot(x, np.cos(x));
Object-oriented interface
The object-oriented interface is available for these more complicated situations, and for when you want more control over your figure.
Rather than depending on some notion of an “active” figure or axes, in the object-oriented interface the plotting functions are methods of explicit Figure and Axes objects.
# First create a grid of plots
# ax will be an array of two Axes objects
fig, ax = plt.subplots(2)
# Call plot() method on the appropriate object
ax[0].plot(x, np.sin(x))
ax[1].plot(x, np.cos(x))
O'Reilly
input file format
Per fer el parsing de python mes facil, lo millor es guardar les dades en format CSV
pi@R4:~/python/pkw/dades/potencies/csv $ cat pot_CSV_2024_07_19.csv
10:11,2617,-82,-2534
un exemple senzill
{ see /home/sebas/dades/python/matplot/4_mpl.py }
def plot_csv ( fnCSV, fnPNG ) :
if beVerbose :
print( ">>> csv in : CSV file {%s}" % fnCSV )
print( ">>> png out : PNG file {%s}" % fnPNG )
fig = plt.figure( figsize=( 1800/my_dpi, 800/my_dpi ), dpi=my_dpi ) # volem PNG de 1800x800 pixels
ax = fig.subplots( ) # "fig" is the canvas, "ax" is the graph
ax.set_facecolor("gray")
# ax.set_xticks( [0, 60, 120, 180, 240, 300, 360, 420, 480, 540, 600, 660, 720, 780, 840, 900, 960, 1020, 1080, 1140, 1200, 1260, 1320, 1380, 1440],
# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24] )
ax.set_xticks( [0, 60, 120, 180, 240, 300, 360, 420, 480, 540, 600, 660, 720, 780, 840, 900, 960, 1020, 1080, 1140, 1200, 1260, 1320, 1380, 1440] ) ;
ax.set_xticklabels( [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24 ] )
X = []
Y = []
n = 0
with open( fnCSV, 'r') as datafile:
plotting = csv.reader(datafile, delimiter=',') # timestamp , potencia , etc , etc
for ROWS in plotting:
n = n + 1
X.append( n )
Y.append( int(ROWS[1]) )
ax.plot( X, Y, color='red', label = 'pProd' ) # specify line colour and graph label
ax.legend() # display label !
ax.fill_between( X, Y, facecolor='green' ) # fill_between uses the colors of the color cycle as the fill color
ax.set_xlabel('eix X (h)')
ax.set_ylabel('eix Y (kWh)', fontweight='bold') # https://www.geeksforgeeks.org/how-to-set-plot-background-color-in-matplotlib/
now = datetime.datetime.now()
szTimestamp = now.strftime("%Y-%m-%d %H:%M:%S %Z")
szSubtitle = szTimestamp + '- CSV line graph, versio {' + szVersion + '}'
plt.suptitle( szSubtitle )
plt.title('dades : {%s}' % fnCSV )
print( "+++ graph in file {%s}" % fnPNG )
plt.savefig( fqfn_fnG )
return 0
funcions clau
Paraules clau :
.figure()
Permet especificar el tamany del PNG - width, height
Per defecte, treu un PNG de 640 x 480 pixels, i te per defecte DPI=100, aixi que es codifica figsize=(6.4, 4.8) per defecte
# plt.figure(figsize=(3.20, 2.40)) # DPI = 100 -> PNG de 320 x 240 pixels
Matplotlib doesn't work with pixels directly, but rather physical sizes and DPI.
If you want to display a figure with a certain pixel size, you need to know the DPI of your monitor.
For example this link
will detect that for you.
Volem produir un PNG de un tamany determinat, diguem 1800 x 800 pixels :
my_dpi = 90
plt.figure( figsize=( 1800/my_dpi, 800/my_dpi ), dpi=my_dpi )
stackoverflow
my_dpi = 96x96 {MARS}, 264x264 {Xiaomi 10}, 144x144 {tablet "Tab A8"}
.plot()
Plot y versus x as lines and/or markers.
.show()
matplotlib will simply output the chart to your viewport when the .show() method is invoked
plt.show()
./z1.py:17: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
you pip install matplotlib and you are trying to display a plot in a GUI window and you do not have a python module for GUI display.
how to display under GUI
- install tkinter :
$ sudo apt-get install python3-tk
- use TkAgg :
import matplotlib
matplotlib.use('TkAgg')
import matplotlib.pyplot as plt # in this order or restart kernel
...
ImportError: Cannot load backend 'TkAgg' which requires the 'tk' interactive framework, as 'headless' is currently running
- installing any of the matplotlib GUI backends like Qt5Agg, GTKAgg, Qt4Agg, etc
$ pip install pyqt5
And here it is, working ...
nicolau@mars:~/sebas/python/smart_devices/termometre/dibuixar$ cat ./1_prova.py
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import matplotlib
matplotlib.use('Qt5Agg') # pip install PyQt5
import matplotlib.pyplot as plt
import numpy as np
y = np.random.random(10)
x = np.linspace(1, len(y), len(y))
plt.plot(x, y)
plt.show(block=True) # no funciona amb "False"
stackoverflow
Use "spider", python IDE for scientists :
$ sudo apt install python3 python3-matplotlib spyder3 ipython3 # Python 3.x,
askubuntu
.savefig()
Save the current figure as an image or vector graphic to a file.
Amb aquesta ajuda
ho treiem a fitxer !
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
import matplotlib.pyplot as plt
data = {'apple': 10, 'orange': 15, 'lemon': 5, 'lime': 20} <<<< la entrada es un diccionari de python
names = list(data.keys())
values = list(data.values())
fig, axs = plt.subplots(1, 3, figsize=(9, 3), sharey=True)
axs[0].bar(names, values)
axs[1].scatter(names, values)
axs[2].plot(names, values)
fig.suptitle('Categorical Plotting')
# plt.show() # mostrar a pantalla ...
plt.savefig('books_read.png') # ... o guardar a fitxer
The available output formats depend on the backend being used.
Onminie, if I run the code
fig, ax = plt.subplots()
sz=fig.canvas.get_supported_filetypes()
print( sz )
... we get {20250105}
{'eps': 'Encapsulated Postscript', 'jpg': 'Joint Photographic Experts Group', 'jpeg': 'Joint Photographic Experts Group',
'pdf': 'Portable Document Format', 'pgf': 'PGF code for LaTeX', 'png': 'Portable Network Graphics', 'ps': 'Postscript',
'raw': 'Raw RGBA bitmap', 'rgba': 'Raw RGBA bitmap', 'svg': 'Scalable Vector Graphics', 'svgz': 'Scalable Vector Graphics',
'tif': 'Tagged Image File Format', 'tiff': 'Tagged Image File Format'}
.ylim()
It is used for setting the y-axis range (or to read it)
Es a dir, si tenim dades en el rang [38..52],
per defecte el "38" sera el començament del eix "Y".
Si volem que comenci a "0", hem de posar .ylim()
Atencio : ha de estar ben aprop de .plot() - mai abans de .figure()
.subplots()
Add an Axes to the current figure or retrieve an existing Axes.
The default call is subplots(nrows=1, ncols=1).
nice article, as
Text inside dollar signs utilizes TeX markup to put variables in italics !!!
capçaleres
Sempre se li ha de dir
escriu un programa python, començant per
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
que faci ...
styles
Creant codi amb Gemini, vols fer servir
OSError: 'seaborn-v0_8-whitegrid' not found in the style library and input is not a valid URL or path;
see `style.available` for list of available styles
Mirem quins estils tenim :
sebas@minie:~/dades/python/matplot$ cat display_available_styles.py
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import matplotlib.pyplot as plt
# List all available styles
styles = plt.style.available
print(styles)
I el resultat es :
sebas@minie:~/dades/python/matplot$ ./display_available_styles.py
['Solarize_Light2', '_classic_test_patch', '_mpl-gallery', '_mpl-gallery-nogrid',
'bmh', 'classic', 'dark_background', 'fast', 'fivethirtyeight', 'ggplot', 'grayscale',
'seaborn', 'seaborn-bright', 'seaborn-colorblind', 'seaborn-dark', 'seaborn-dark-palette',
'seaborn-darkgrid', 'seaborn-deep', 'seaborn-muted', 'seaborn-notebook', 'seaborn-paper',
'seaborn-pastel', 'seaborn-poster', 'seaborn-talk', 'seaborn-ticks', 'seaborn-white', 'seaborn-whitegrid', 'tableau-colorblind10']
proves
Fem proves a MARS - obtenim PNG de 640 x 480 pixels
/home/nicolau/sebas/python/matplot/2_plot.py -> ./line_graph.png
I amb dades reals treballem a
(r0) /home/pi/python/fronius/z_extract_temp_from_json_and_do_graph.py -> ./d_temperatures/line_graph_2022-08-04.png
Compte : al PiZero hem de fer servir python3, vist que no es pot instalar matplotlib per python2
R4 : llegim texte temperatures i generem PNG :
pi@R4:~/python/termometre $ cat 2_plot_temp.py
# every line is in this format :
# 09:45:02 - T 193 , H 52
# 0 1 2 3
for line in open(inFN, 'r'):
lines = [i for i in line.split()]
print("0 = {%s}" % lines[0])
print("3 = {%s}" % lines[3])
x.append(lines[0])
y.append(int(lines[3]))
plt.yticks(y)
plt.plot(x, y, marker = 'o', c = 'g') #
plt.plot()
# plt.show()
plt.savefig(outFN)
backend
To support all use cases, matplotlib can target different outputs, and each of these capabilities is called a backend;
the "frontend" is the user facing code, i.e., the plotting code,
whereas the "backend" does all the hard work behind-the-scenes to make the figure.
Matplotlib backend relies on some backend to actually render the plots.
The default backend is the agg backend.
This backend only renders PNGs.
Docs for Matplotlib backend.
So, if you want to generate an image but not to display it, use this code :
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
…
plt.savefig(outFN)
matplotlib problems
Jul 21 18:34:56 pi0 fronius-nodejs[10603]: *** 2024/07/21, 18:34:56 - --- python POT RC (Error: /home/pi/njs/fronius/5_potencia_csv_a_png.py:45: UserWarning:
Jul 21 18:34:56 pi0 fronius-nodejs[10603]: This call to matplotlib.use() has no effect because the backend has already
Jul 21 18:34:56 pi0 fronius-nodejs[10603]: been chosen; matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
Jul 21 18:34:56 pi0 fronius-nodejs[10603]: or matplotlib.backends is imported for the first time.
Jul 21 18:34:56 pi0 fronius-nodejs[10603]: The backend was *originally* set to 'TkAgg' by the following code:
Jul 21 18:34:56 pi0 fronius-nodejs[10603]: File "/home/pi/njs/fronius/5_potencia_csv_a_png.py", line 42, in <module>
Jul 21 18:34:56 pi0 fronius-nodejs[10603]: import matplotlib.pyplot as plt
Solucio :
pi@pi0:~/njs/fronius $ cat 5_potencia_csv_a_png.py
import matplotlib # 20240721
matplotlib.use('Agg') # 20240721, Error: _tkinter.TclError: no display name and no $DISPLAY environment variable
# matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
# ... or matplotlib.backends is imported for the first time.
import matplotlib.pyplot as plt #
URLs matplotlib
my python projects
- recollir sys info - go
- enviar missatge a telegram - go
- manage GPIO pins - go
- web server - go
- send email, SMTP - go
- read email, IMAPLIB - go
- manage images and webcam, openCV - go
- sounds - go
- I2C and TC74 - go
- blog - go
- AWS access
exercise : get sys info into text/html file + ftp
Recollir informacio del sistema, escriure un fitxer HTML i enviar-lo per FTP
Resultat : PiZero ,
Odin ,
by //MARS:/home/nicolau/sebas/python/tinet/R0/enviar_ip_a_tinet.py
Send message to Telegram bot
Enviar texte a un bot - tg.htm
Manage Raspberry GPIO using python
What is RPi.GPIO?
It’s an easy way of controlling the Pi’s General Purpose Input Output ports in python.
Display RPi.GPIO version
#!/usr/bin/env python3
import RPi.GPIO as GPIO # sudo apt-get install python3-rpi.gpio
print( "GPIO versio ", GPIO.VERSION ) # mostrar la versio de GPIO
print( "RPi revision ", GPIO.RPI_REVISION ) # mostrar la revisio del Raspberry Pi -
url
Basic GPIO code
sag@odin:~/gpio_python $ cat bucle_sortida_pin37.py
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# executem amb "sudo ./bucle_sortida_pin37.py"
import RPi.GPIO as GPIO
import time
GPIO.setmode(GPIO.BOARD) # escollim numeracio "com a la placa"
SORTIDA = 37 # com sortida fem servir el penultim pin de l'esquerra (see command "pinout") - GPIO26, pin 37
GPIO.setup( SORTIDA, GPIO.OUT ) # programem el pin com de "sortida"
i = 0
retard = 2
while i < 100: # white True:
i = i + 1
print( 'i = ', i ) # mostrem activitat
print( '(1) encenem el llum' )
GPIO.output(SORTIDA, GPIO.HIGH) # output "ON"
print( '(2) esperem (%s) sg' % retard )
time.sleep( retard ) # wait(2 sg)
print( '(3) apaguem el llum' )
GPIO.output(SORTIDA, GPIO.LOW) # output "OFF"
print( '(4) esperem (%s) sg' % retard )
time.sleep( retard ) # wait(2 sg)
print( '(99) netejem i acabem' )
GPIO.cleanup()
"channel in use" warning
If we get
RuntimeWarning: This channel is already in use, continuing anyway. Use GPIO.setwarnings(False) to disable warnings.
We have to improve our code to execute "cleanup()" on all cases :
retard = 2 # nombre de segons q esperem
try:
while True: # forever loop ; exit using SIGINT
i = i + 1
print( 'i = ', i ) # mostrem activitat
print( '(1) encenem el llum' )
GPIO.output(SORTIDA, GPIO.HIGH)
print( '(2) esperem (%i) segons' % retard )
time.sleep( retard ) # wait()
print( '(3) apaguem el llum' )
GPIO.output(SORTIDA, GPIO.LOW)
print( '(4) esperem (%s) segons' % retard )
time.sleep( retard )
except KeyboardInterrupt:
print( '--- tenim CTRL C' )
except:
print( '--- --- ALL other exceptions' )
finally:
print( '>>> netejem i acabem' )
GPIO.cleanup()
Read raspi.tv
web server
run one-liner (python3)
From python v3 we can go one-liner :
$ python3 -m http.server 8084 # files are served from current directory
$ python3 -m http.server 8084 --directory /tmp/prova/ # from python v 3.7 up
afternerd
Per exemple, si volem mirar un codi HTML + JavaScript local, (MARS+njs+pere)
obtindrem un missatge a la consola com :
Access to fetch at 'file:///home/nicolau/sebas/njs/pere/info.html' from origin 'null'
has been blocked by CORS policy:
Cross origin requests are only supported for protocol schemes:
http, data, chrome, chrome-extension, chrome-untrusted, https.
Then, we start a python server and proceed :
nicolau@mars:~/sebas/njs/pere$ python3 -m http.server
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
127.0.0.1 - - [14/Aug/2022 09:06:32] "GET / HTTP/1.1" 304 -
127.0.0.1 - - [14/Aug/2022 09:06:32] "GET /styles.css HTTP/1.1" 304 -
127.0.0.1 - - [14/Aug/2022 09:06:32] "GET /main.js HTTP/1.1" 304 -
from code
Volem un web server lleugerament semblant a un node, es a dir que
- serveixi pagines estatiques
- faci una foto i la envii (fitxer on-request) si es prem un boto
- respongui a requeriments del client, com "a quina temperatura tens la GPU ?"
We can code :
import http.server #
docs
import socketserver #
docs
PORT = 8083
Handler = http.server.SimpleHTTPRequestHandler
with socketserver.TCPServer(("", PORT), Handler) as httpd:
print("serving at port", PORT)
httpd.serve_forever()
cgi-bin python web server
We code :
sag@odin:~/python/web_server $ cat /home/sag/python/web_server/cgi/my_cgi_server.py
#!/usr/bin/env python3
# web server
# run with "./my_cgi_server.py"
# sudo adduser sebas gpio
# access using http://myraspiodin.hopto.org:8123/cgi-bin/one.py
# https://stackoverflow.com/questions/13065306/how-to-execute-python-script-on-the-basehttpserver-created-by-python
from http.server import CGIHTTPRequestHandler, HTTPServer
my_Host = '192.168.1.123'
my_Port = 8123
print( "*** web server, ip (", my_Host, "), port (", my_Port, ")." )
handler = CGIHTTPRequestHandler
handler.cgi_directories = ['/cgi-bin', '/htbin'] # this is the default
server = HTTPServer(( my_Host, my_Port ), handler)
server.serve_forever()
To start, we serve this file :
sag@odin:~/python/web_server/cgi $ cat index.htm
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title> *** mon Index *** </title>
</head>
<body>
<hr>
<p> <a href="http://myraspiodin.hopto.org:8123/cgi-bin/one.py"> Encendre </a>
<p> .
<p> << href="http://myraspiodin.hopto.org:8123/cgi-bin/two.py"> Apagar </a>
<r>
</body>
</html>
And the action is performed here :
sag@odin:~/python/web_server/cgi/cgi-bin $ cat one.py
#!/usr/bin/env python3
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BOARD) # escollim numeracio "com a la placa"
SORTIDA = 37 # com sortida fem servir el penultim pin de l'esquerra (see command "pinout") - GPIO26, pin 37
GPIO.setup( SORTIDA, GPIO.OUT ) # programem el pin com de "sortida"
GPIO.output(SORTIDA, GPIO.HIGH) # set pin HIGH
print( 'Content-type: text/html\n')
print( '<title> +++ ON +++ </title>')
print( ' <body> <hr> ' )
print( ' <p> +++ now LED is ON +++ ' )
print( ' <p> Per apagar <a href="http://myraspiodin.hopto.org:8123/cgi-bin/two.py"> pica aqui </a> </p> ' )
print( ' <hr> </body>')
And here :
sag@odin:~/python/web_server/cgi/cgi-bin $ cat two.py
#!/usr/bin/env python3
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BOARD) # escollim numeracio "com a la placa"
SORTIDA = 37 # com sortida fem servir el penultim pin de l'esquerra (see command "pinout") - GPIO26, pin 37
GPIO.setup( SORTIDA, GPIO.OUT ) # programem el pin com de "sortida"
GPIO.output(SORTIDA, GPIO.LOW) # set pin LOW
print('Content-type: text/html\n')
print('<title> --- OFF --- </title>')
print( ' <body> <hr> ' )
print( ' <p> --- now LED is OFF --- ' )
print( ' <p> Per encendre <a href="http://myraspiodin.hopto.org:8123/cgi-bin/one.py"> pica aqui </a> </p> ' )
print( ' <hr> </body>')
python webserver channel warning
We get this interesting runtime warning :
/home/sag/python/web_server/cgi/cgi-bin/one.py:6: RuntimeWarning:
This channel is already in use, continuing anyway. Use GPIO.setwarnings(False) to disable warnings.
GPIO.setup( SORTIDA, GPIO.OUT ) # programem el pin com de "sortida"
Solution : code cleanup() as indicated here
python web server links
Some URLs :
using Google Sheets from python
- login to Google account {pocaporpere}
- set up a Google Cloud platform project :
- enter Google Cloud console
- "Select a project" + "New project" {top right}
- Projet Name := "minie python 2 Gdatasheets" + "Create"
- enable Google Sheets API :
- search for "google sheets api" in top center box
- first entry is "Google Sheets API" + "Google Enterprise API" - click on it
- "Google Sheets API" page comes up - click "Enable"
- no enable Cloud Natural Language API :
- search for "cloud natural language api" in top center box
- click "Enable"
- Cloud Natural Language API requires a project with a billing account
- create API key :
- open top-left 3-lines and select "APIs and services", then "Credentials"
- click "create credentials" - here we have 3 options :
- API key -
- OAuth client ID -
- Service account - get JSON file
- + "API key" - it is used to authenticate to the 2 API's
With this authentication our API will only work with "public" Google Sheets
To use "private" Google Sheets we'd need to use OAuth authentication (or Service Account ?)
save the key for later use
- access using Google Service Account (JSON file)
- go to Google Cloud Console
- ensure the correct project is selected from the dropdown menu at the top of the page
- navigate to Service Accounts in the navigation menu (the hamburger icon), go to IAM & Admin > Service Accounts
- click "+ Create Service Account" nom "servei-isins"
- create the JSON key :
- Once you are on the Service Account details page, click the Keys tab.
- Click the Add Key dropdown menu, and select Create new key.
- In the dialog box that appears, select the JSON key type.
- Click Create.
- download the file {20251215 - isin2gss-df6e78707986.json}
- Service Account is shared with the Sheet:
Find the Client Email in your isin2gss-df6e78707986.json file (it looks like [account-name]@[project-id].iam.gserviceaccount.com).
Go to your Google Spreadsheet, click the Share button, and share the spreadsheet with that specific Service Account email, giving it Editor permission.
- enable Google API : visit here
- to connect to Google data, we'll be using Google libraries, built in toolkits
$ pip3 install google-api-python-client google-cloud-language
Jo faig servir "software manager" + "python3-googleapi" v 1.7.11-4 & python3-googleapi-samples
destinacio a minie : /usr/share/doc/python3-googleapi-samples/examples
Homepage
- create a Spreadsheet, and name it "miniepython2GS" :
Here is the
private spreadsheet URL
and ID is "1W070QyiBSOP_bEwCIyO2wziAYoAV3F4z5QgeIkZq5E0", owned by "pocaporpere"
Public URL
Using AI we get this code : chatGPT write GSS -> sebas@minie:~/dades/python/gugel_sheets$./4_chat_gpt.py &
details (install, errors, ...)
Basic concepts
To write to a sheet, you need the spreadsheet ID, the range of cells in A1 notation, and the data you wish to write within an appropriate request body object.
Updates require a valid ValueInputOption parameter :
- "RAW" : the input is not parsed and is inserted as a string
- "USER_ENTERED" : the input is parsed exactly as if it were entered into the Sheets UI
Single quotes are required for sheet names with spaces, special characters, or an alphanumeric combination.
Cell
The "Cells" resource represents each cell, but it doesn't have a unique ID value.
Instead, row and column coordinates identify the cells.
API overview, concepts
Basic code
developers.google
read cell values
Simple read :
sebas@minie:~/dades/python/gugel_sheets$ cat 2_rw_gss.py
from google.oauth2 import service_account # "pip3 install google" + "pip3 install --upgrade google-api-python-client"
from googleapiclient.discovery import build
spreadsheet_id = "1B987Qyi333P_b647IyOquiquioAV3F4z5QgeIkZq4Ez"
credentials = service_account.Credentials.from_service_account_file("key.json", scopes=["https://www.googleapis.com/auth/spreadsheets"]) # set up credentials
service = build("sheets", "v4", credentials=credentials) # create Sheets API client
request = service.spreadsheets().get(spreadsheetId=spreadsheet_id, ranges=[], includeGridData=False)
sheet_props = request.execute()
print(sheet_props["properties"]["title"])
hackernoon
update() cell values
Basic write :
sebas@minie:~/dades/python/gugel_sheets$ cat 2_rw_gss_pere.py
from google.oauth2 import service_account # "pip3 install google" + "pip3 install --upgrade google-api-python-client"
from googleapiclient.discovery import build
spreadsheet_id = "1B987Qyi333P_b647IyOquiquioAV3F4z5QgeIkZq4Ez"
credentials = service_account.Credentials.from_service_account_file("key.json", scopes=["https://www.googleapis.com/auth/spreadsheets"]) # set up credentials
service = build("sheets", "v4", credentials=credentials) # create Sheets API client
now = datetime.datetime.now()
szTemps = now.strftime("%Y-%m-%d %H:%M:%S")
range_name = "Sheet1!B2:B2" # starting and ending cell coordinates
new_values = [[szTemps]] # Values are intended to be a 2d array
body = {"values": new_values}
result = service.spreadsheets().values().update(
spreadsheetId=spreadsheet_id, range=range_name,
valueInputOption="USER_ENTERED", body=body).execute()
range_name = "Sheet1!B4:C6" # starting and ending cell coordinates
new_values = [['Name', 'Age'], ['John', '30'], ['Jane', '25']]
body = {"values": new_values} # values are intended to be a 2d array
result = service.spreadsheets().values().update(
spreadsheetId=spreadsheet_id, range=range_name,
valueInputOption="USER_ENTERED", body=body).execute()
coefficient.io
If successful, the response body contains an instance of
UpdateValuesResponse
com fer "public" un Google SpreadSheet
To make a document, spreadsheet, or presentation available for a large audience to see, publish the file.
After you publish your file you can send a new URL to anyone or embed into your website.
Any changes you make to the original document will be updated in the published version.
- go Sheets
and login()
- at the top left, click "File" and then "Share" and then "Publish to web"
- copy "public" sheet URL :
Fronius from R4
or
xatGPT
support.google.com
la meva Google SpreadSheet amb la potencia diaria del Fronius
El R4 escriu cada nit a aquesta pagina :
write to GSS using chatGPT code
My question was "acces to google spreadsheets from python"
- instalem el modul :
sebas@minie:~/dades/python/gugel_sheets$ python3 -m pip install gspread
Defaulting to user installation because normal site-packages is not writeable
Collecting gspread
Downloading gspread-6.1.4-py3-none-any.whl (57 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.6/57.6 KB 49.7 kB/s eta 0:00:00
Collecting google-auth-oauthlib>=0.4.1
Downloading google_auth_oauthlib-1.2.1-py2.py3-none-any.whl (24 kB)
Collecting google-auth>=1.12.0
Downloading google_auth-2.38.0-py2.py3-none-any.whl (210 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 210.8/210.8 KB 59.8 kB/s eta 0:00:00
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/lib/python3/dist-packages (from google-auth>=1.12.0->gspread) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/lib/python3/dist-packages (from google-auth>=1.12.0->gspread) (0.2.1)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/lib/python3/dist-packages (from google-auth>=1.12.0->gspread) (5.0.0)
Collecting requests-oauthlib>=0.7.0
Downloading requests_oauthlib-2.0.0-py2.py3-none-any.whl (24 kB)
Requirement already satisfied: requests>=2.0.0 in /usr/lib/python3/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib>=0.4.1->gspread) (2.25.1)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/lib/python3/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib>=0.4.1->gspread) (3.2.0)
Installing collected packages: requests-oauthlib, google-auth, google-auth-oauthlib, gspread
Successfully installed google-auth-2.38.0 google-auth-oauthlib-1.2.1 gspread-6.1.4 requests-oauthlib-2.0.0
- error classic :
gspread.exceptions.SpreadsheetNotFound: <Response [200]>
Solucio :
you need to share the spreadsheet with the "client_email" mail id
open the spreadsheet and then click on the "share" button (top right) and then just paste the "client_email" (email id) from JSON
stackOverflow,
gspread Docs on "Auth"
Posem "comptexatgpt@xatgpt.iam.gserviceaccount.com" i piquem "Compartir"
- error curios :
gspread.exceptions.APIError: APIError:
[400]: Invalid value at 'data.values[0]'
(type.googleapis.com/google.protobuf.ListValue), "13:05:34 al cap de munt de tot"
ValueRange :
values[] array (ListValue format)
The data that was read or to be written.
This is an array of arrays, the outer array representing all the data and each inner array representing a major dimension.
Each item in the inner array corresponds with one cell.
Solucio - per "append_row()", el texte ha de ser un array :
szTexte = [szHMS + " al cap de munt de tot"]
- error similar :
gspread.exceptions.APIError: APIError: [400]: Invalid values[1][0]: list_value {
values {
string_value: "txt a lloc determinat"
}
}
Solucio - per "update_cell()" el texte ha de ser un string
detalls de gspread
gspread is a Python API for Google Sheets -
under the hood, gspread uses Google Sheets API v4
gspread
homepage
(exemples),
github
(exemples)
Sample :
import gspread # pip install gspread
gc = gspread.service_account() # authenticate using a service account
worksheet_list = gc.worksheets() # get a list of all worksheets
wks = gc.open("Where is the money ?").sheet1 # open a worksheet from a spreadsheet
wks.update([[1, 2], [3, 4]], "A1") # update a range of cells using the top left corner address
wks.update_acell("B42", "it's down there somewhere.") # update a single cell
wks.format('A1:B1', {'textFormat': {'bold': True}}) # format the header
codi python amb gspread
Minie escriu on-demand a aquesta pagina :
I el accés es fa així :
sebas@minie:~/dades/python/gugel_sheets$ cat 4_chat_gpt_ignasi.py
ISIN i python
An ISIN is a permanent, unique 12-character code used to identify a specific security, such as a stock, bond, or fund.
What is the value (e.g., stock price) for a security today?
You would typically use a Ticker Symbol for this in python.
A Ticker Symbol is specific to a single exchange.
To get the stock price (value) using python, you would typically use a library like yfinance or pandas_datareader.
Most free and popular Python libraries like yfinance only accept the Ticker Symbol (e.g., AAPL or QDVW.DE), not the ISIN directly.
Reading the current stock price (value) of an ISIN using Python requires a crucial intermediary step: Symbology Mapping.
The most robust way is by using a dedicated financial API.
Two good options that support ISIN lookups are Financial Modeling Prep (FMP) and Finnhub (both often have free or developer tiers).
Financial Modeling Prep (FMP): https://site.financialmodelingprep.com/ + API Documentation: https://site.financialmodelingprep.com/developer/docs
Login (abel) ->
To authorize your requests, add ?apikey=YOUR_API_KEY at the end of every request.
API key : G4XwnJmcUpFY8RWvywRZhdnpQ7aVkwNo
Finnhub: https://finnhub.io/ + API Documentation: https://finnhub.io/docs/api/introduction
dubtes que tinc amb Google Sheets
- com veure si el meu Google Sheets es "public" o "private" ?
- Google spreadsheets can be shared with an email - how to know who am I sharing this spreadsheet ?
chatGPT - to see who you're sharing a Google Spreadsheet with, follow these steps:
- Open the Google Spreadsheet you want to check.
- Click on the "Share" button in the top-right corner of the screen (it's a blue button).
- In the "Share with people and groups" window that appears, you'll see a list of all the people you've shared the document with. This will include:
- Emails of people who have access to the sheet.
- The permission level (Viewer, Commenter, Editor).
- You may also see whether it's shared with a specific group or link (if anyone with the link can access it).
- If you scroll down, you can also see the link sharing settings (e.g., anyone with the link, restricted access, etc.).
This window shows a list of everyone with access to your spreadsheet, so you can easily see who you're sharing it with.
gspread errors
- Sheets API v4 introduced Usage Limits,
as of this writing, 300 requests per 60 seconds per project, and 60 requests per 60 seconds per user.
When your application hits that limit, you get an APIError 429 RESOURCE_EXHAUSTED.
- API exceptions
URL's to Google Sheets from python
Google Custom Search API
La Google Custom Search API (també coneguda com a JSON Custom Search API)
és la solució oficial de Google per fer cerques de text o d'imatges de forma programàtica.
A diferència del scraping (que és el que feies amb DuckDuckGo), aquesta API està dissenyada per a desenvolupadors,
és estable i no et bloquejarà per "Ratelimit" si et mantens dins dels marges.
Com funciona?
Per utilitzar-la, no busques directament a "tot Google",
sinó que crees un Motor de Cerca Personalitzat (Programmable Search Engine)
i després hi connectes el teu script mitjançant una clau de seguretat.
Què necessites per començar?
Per fer-la servir al teu programa Python, necessites dues coses:
- API Key: Una clau alfanumèrica que identifica el teu projecte a Google Cloud.
- Search Engine ID (CX): Un identificador que crees a la consola de Programmable Search Engine.
Tip: En configurar-lo, has d'activar l'opció de "Cerca d'imatges" i seleccionar l'opció de "Cercar a tota la web".
Costos
Gratis: Fins a 100 cerques al dia. Per a un script personal o de prova, sol ser suficient.
Pagament: Si necessites més, costa uns $5 per cada 1.000 cerques addicionals.
Configuracio
Per fer servir la Google Custom Search API, has de seguir dos processos:
un a la consola de Google Cloud (per la clau)
i un altre al configurador de cercadors (per l'ID del motor).
Clau API
- Ves a Google Cloud Console {usr Jhngr}
- Si no tens cap projecte, crea'n un de nou (clica al desplegable de dalt a l'esquerra -> New Project).
Posa-li un nom (ex: "projecte per cercar cares").
- Al cercador superior, escriu "Custom Search API" i clica-hi.
Clica el botó Enable (Habilitar).
- Un cop habilitada la AOI, ves a la pestanya lateral "Credentials".
Clica a + Create Credentials -> API Key.
- Apareixerà una finestra amb la teva clau (comença per AIza...). Copia-la i guarda-la bé.
Search Engine ID
- Ves a Programmable Search Engine
Clica a Add (Afegir).
Posa-li un nom (ex: "cercador de cares").
- Al camp "What to search?", selecciona "Search the entire web".
- Un cop creat el motor (engine), ves a la configuració (Customize) del cercador que acabes de fer.
Molt important: Busca l'opció Image Search (Cerca d'imatges) i activa-la (ON).
Busca el camp Search engine ID i copia el codi que apareix.
El neu buscador de cares en python
sebas@minie:~/dades/ai/cares$ ./5_fer.py
Llegint fitxer 'noms.txt'
Iniciant la cerca per a 2 persones...
Buscant cara de: Raphinha
Buscant cara de: Vladimir Putin
Feina feta:
- Creat 'cares.html' amb la taula de resultats.
- Creat './urls.txt' amb les URLs de les imatges.
send email - SMTP (and postfix)
Requires a Google low security account
$ cat //MARS//home/nicolau/sebas/python/send_email/alby.py
#!/usr/bin/env python3
#
# send mail: to whoever, from whoever you think of, and from wherever you want to be :-)
#
import sys
# import socket
import smtplib
if __name__ == '__main__':
print( sys.argv )
if len(sys.argv) != 5:
print( "format <from> <to> <subject> <body>\n" )
sys.exit (-1)
else:
smtpconn = smtplib.SMTP ('localhost')
print( "Sending mail from:", sys.argv[1] )
print( "Sending mail to:", sys.argv[2] )
print( "Subject:", sys.argv[3] )
print( "Body:", sys.argv[4] )
print( "in progress..." )
smtpconn.sendmail (sys.argv[1], sys.argv[2], "Subject: " + sys.argv[3] + "\r\n" + sys.argv[4])
print( "done" )
sys.exit (0)
read email using poplib - POP3 protocol client
specs
Minimal example : (from here)
import getpass, poplib
M = poplib.POP3('localhost')
M.user(getpass.getuser())
M.pass_(getpass.getpass())
numMessages = len(M.list()[1])
for i in range(numMessages):
for j in M.retr(i+1)[1]:
print(j)
A more detailed code :
sebas@minie:~/dades/python/email/POP_read_mail$ cat gmx_neus.py
#!/usr/bin/env python
# POP3 client
# http://net-informations.com/python/net/pop3.htm
# https://www.code-learner.com/python-use-pop3-to-read-email-example/
import poplib
import sys
from email.parser import Parser
pop3server = 'pop.gmx.com'
username = 'neus.bertolin@gmx.com'
password = 'neus_pwd'
pop3server = poplib.POP3_SSL(pop3server) # open connection
print( pop3server.getwelcome() ) # show welcome message
pop3server.user(username)
pop3server.pass_(password)
# stat() function return email count and occupied disk size
print( 'Messages: %s. Size: %s' % pop3server.stat() )
# list() function return all email list
resp, mails, octets = pop3server.list()
print( 'email list %s.' % mails )
pop3info = pop3server.stat() # access mailbox status - result is 2 integers: (message count, mailbox size)
mailcount = pop3info[0] # get message count
print( "Total no. of Email : ", mailcount )
print ("\n=== Start Reading Messages")
for i in range(mailcount):
print( "\n+++ email index (", i, ")." )
resp, lines, octets = pop3server.retr( i+1 )
# lines stores each line of the original text of the message
# so that you can get the original text of the entire message use the join function and lines variable.
msg_content = b'\r\n'.join(lines).decode('utf-8')
# now parse out the email object.
msg = Parser().parsestr(msg_content)
# get email from, to, subject attribute value.
email_from = msg.get('From')
print('From ' + email_from)
email_to = msg.get('To')
print('To ' + email_to)
email_subject = msg.get('Subject')
print('Subject ' + email_subject)
pop3server.quit()
sys.exit()
imap
Mind imap tools
read email using imaplib - IMAP4 protocol client
To access a imap server we can use this code that provides access to imaplib :
\\w500\python\read_emails> type imap.py
#!/usr/bin/env python
import imaplib
import email
import sys
# set default values
my_user = 'myuser@gmail.com'
my_pwd = 'mypwdmypwd'
my_box = 'inbox'
my_server = 'pop.gmail.com'
# use command line params if available
print( '++++++++++++++++++++ Got ('+ str( len(sys.argv)) +') params.' )
if len(sys.argv) > 1: my_user = str( sys.argv[1] )
if len(sys.argv) > 2: my_pwd = str( sys.argv[2] )
if len(sys.argv) > 3: my_box = str( sys.argv[3] )
if len(sys.argv) > 4: my_server = str( sys.argv[4] )
# display values to use
print( '++++++++++++++++++++++++++++++++++++++++ User (' + my_user + ').' )
print( '++++++++++++++++++++++++++++++++++++++++ Pwd (' + my_pwd + ').' )
print( '++++++++++++++++++++++++++++++++++++++++ Box (' + my_box + ').' )
print( '++++++++++++++++++++++++++++++++++++++++ Server (' + my_server + ').' )
print( '(1) *** connect to server (', my_server, ').' )
mmail = imaplib.IMAP4_SSL( my_server )
print( '(2) *** login to user (', my_user, ').' )
mmail.login( my_user, my_pwd )
print( '(3) *** list of folders or gmail labels' )
mmail.list()
for i in mmail.list()[1]:
l = i.decode().split(' "/" ')
print(l[0] + " = " + l[1])
# Hem de anar al sub-folder indicat
print( '(4) *** connect to specific folder (', my_box, ').' )
stat, nummsg = mmail.select( my_box )
szNumMsg = str(nummsg)
newMsgNum = szNumMsg.split("'")[1]
print( '(5) tenim ' + str(newMsgNum) + ' msg' )
result, data = mmail.search( None, "ALL" )
szData = str( data )
newData = szData.split("'")[1]
print( '(6) select (', newData, ') msg' )
ids = data[0] # data is a list.
# newIds = ids.split("'")[1]
print( '(7) IDS (', ids, ').' )
id_list = ids.split() # ids is a space separated string
print( '(8) ID LIST (', id_list, ').' )
for num in id_list:
print( '(9) Msg ID is (', num, newMsgNum, ').' )
typ, msg_data = mmail.fetch( num, "(RFC822)" ) # fetch the email body (RFC822) for the given ID
email_content = msg_data[0][1]
for response_part in msg_data:
if isinstance( response_part, tuple):
msg = email.message_from_string( email_content.decode() )
email_from = msg['from']
email_subj = msg['subject']
print( '(10) subject is (', email_subj, ').' )
print( 'close' )
mmail.close()
print( 'logout - shutdown connection to server' )
mmail.logout()
To run this python, we shall use
- "imap.py myuser@gmail.com mypwd folder_name imap.gmail.com"
- "imap.py myuser@yandex.com mypwd folder_name imap.yandex.com"
gmail account has to have less secure apps access enabled !
gmx account has to be accessed with "user@gmx.com" format as userid
delete email using imaplib
Here is the working code :
sebas@minie:~/dades/python/email/IMAP_read_mail/esborrar$ cat delete_files_in_folder.py
#!/usr/bin/python3
# https://stackoverflow.com/questions/3180891/imap-how-to-delete-messages
from imap_tools import MailBox
import imaplib
import sys
# set default values
my_user = 'char@gmail.com'
my_pwd = 'wrdfwjygvn' # app pwd !
my_box = 'inbox'
my_server = 'pop.gmail.com'
# use command line params if available
print( '++++++++++++++++++++ Got ('+ str( len(sys.argv)) +') params.' )
if len(sys.argv) > 1: my_user = str( sys.argv[1] )
if len(sys.argv) > 2: my_pwd = str( sys.argv[2] )
if len(sys.argv) > 3: my_box = str( sys.argv[3] )
if len(sys.argv) > 4: my_server = str( sys.argv[4] )
# display values to use
print( '++++++++++++++++++++++++++++++++++++++++ User (' + my_user + ').' )
print( '++++++++++++++++++++++++++++++++++++++++ Pwd (' + my_pwd + ').' )
print( '++++++++++++++++++++++++++++++++++++++++ Box (' + my_box + ').' )
print( '++++++++++++++++++++++++++++++++++++++++ Server (' + my_server + ').' )
# from imap_tools import MailBox
box = imaplib.IMAP4_SSL( my_server, 993 )
box.login( my_user, my_pwd )
stat, nummsg = box.select( my_box, readonly=False )
total_msg = nummsg[0].decode('utf-8')
print(f"() Tenim {total_msg} missatges")
szNumMsg = str(nummsg)
newMsgNum = szNumMsg.split("'")[1]
print( '() tenim ' + str(newMsgNum) + ' msg' )
typ, data = box.search(None, 'ALL')
print( '>>> typ is {}'.format(typ) )
print( '>>> data is {}'.format(data) )
for num in data[0].split():
print( '>>> moure a BIN {}/{}'.format(num, newMsgNum) )
box.copy(num, '[Gmail]/Bin')
print( '>>> delete {}/{}'.format(num, newMsgNum) )
box.store(num, '+FLAGS', '\\Deleted')
box.expunge()
box.close()
box.logout()
imaplib links
openCV
OpenCV és una llibreria de programació de codi obert dirigida principalment a la visió per computador en temps real,
desenvolupada per la divisió russa d'Intel en el centre de Nijni Nóvgorod.
OpenCV homepage,
wiki
Intro by Albert {2016.06.02} :
OpenCV + "contours" + "webcam"
Per exemple tens el programa "python" corrent, acostes un taulell d'escacs a la webcam, i veus a la pantalla els poligons que trobi el python en temps real.
T'estalvies una burrada de hores, a part que es molt divertit i avances molt depressa.
Una mica de codi python :
nicolau@mars:~/sebas/python/openCV$ ls
4 -rwxr--r-- 1 nicolau nicolau 325 Apr 16 10:12 0_versio_opencv.py
4 -rwxr--r-- 1 nicolau nicolau 810 Apr 16 10:13 1_captura_webcam_a_crt.py
4 -rwxr--r-- 1 nicolau nicolau 985 May 24 2020 2_webcam_contours.py
4 -rwxr--r-- 1 nicolau nicolau 53 May 9 2020 3_veure_contingut_cv2.py
OpenCV profiling
OpenCV profiling
Runtime options can be specified via environment variables.
- OPENCV_TRACE - enables OpenCV tracing. By default tracing is turned off. It automatically enables on active ITT backend on startup.
- OPENCV_TRACE_LOCATION - storage path to store OpenCV trace.
- OPENCV_TRACE_DEPTH_OPENCV - specify trace depth of OpenCV functions
0 - limit is disabled
1 - trace OpenCV entry points only, without inner details (default)
- OPENCV_TRACE_MAX_CHILDREN - limit number of traced children (useful to ignore functions with large loops)
- OPENCV_TRACE_MAX_CHILDREN_OPENCV - limit number of traced children for OpenCV calls
- OPENCV_TRACE_SYNC_OPENCL - force clFinish calls in the end of OpenCV regions (performance impact, default value is 0)
openCV install under Ubuntu
Tutorial
$ sudo apt-get install python-opencv
(20200420) I prefer webcam with Motion + browser
- sudo apt update
- sudo apt install python-opencv {lets use python 2 by now}
- verify the installation :
nicolau@nicolau-desktop:~/sebas/python/openCV$ python -c "import cv2; print(cv2.__version__)"
3.2.0
openCV demo
Run contours.py
//MARS//home/nicolau/sebas/python/openCV/1_captura_webcam_a_crt.py -> OK
//MARS//home/nicolau/sebas/python/openCV/2_webcam_contours.py -> OK
url
openCV install under Windows
Windows Install -
opencv-3.1.0.exe {20160603, win 2k8}
En executar el EXE es crea la carpeta
Compte amb els projectes :
The systems PATH contains a list of folders where DLLs can be found.
So we include it in system PATH :
C:\sebas\miscosas\albert\python\webcam> echo %PATH%
C:\opencv;C:\python27;C:\ProgramData\Oracle\Java\javapath;c:\eines;C:\Windows\system32; ...
The easiest way to install OpenCV on Windows is to simply copy the cv2.pyd file into python's site-packages directory
(e.g. C:\python27\Lib\site-packages).
Here is the complete
sequence.
Verify openCV install
Code
import cv2
print "### CV2 version I have is " + cv2.__version__
Run it
CV2 version I have is 3.1.0
Display openCV contents
Supose we have this error : AttributeError: 'module' object has no attribute 'namedWindow'
Display CV2 contents using :
- enter python command line : python
- load cv2 : import cv2
- list cv2 contents : help(cv2) -> 2.482 linies !
See //MARS/home/nicolau/sebas/python/openCV/3_veure_contingut_cv2.py
Conclusion : names are Case Sensitive and the exact name is "namedWindow"
openCV functions
opencv documentation !
User Interface te :
contours.py
"contours.py" fa servir aquestes funcions :
cap = cv2.VideoCapture(0) # connecto to webcam
cap.isOpened() #
ret, frame = cap.read()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray,(5,5),0)
ret, thresh_img = cv2.threshold(blur,91,255,cv2.THRESH_BINARY)
contours = cv2.findContours(thresh_img,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)[-2]
cv2.drawContours(frame, [c], -1, (0,255,0), 3)
cv2.imshow('frame title',frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
cap.release()
cv2.destroyAllWindows()
openCV errors
If we get this error :
cv2.error: /build/opencv-VF5Hiu/opencv-2.4.9.1+dfsg/modules/highgui/src/loadsave.cpp:318:
error: (-2) could not find a writer for the specified extension in function imwrite_
... it means we specified a destination filename with a wrong extension.
Accepted
are ".png", ".jpeg" ...
webCam project
Descripció del nostre projecte :
- run Android program to capture a video containing (aprox, configurable)
- 1 frame every 5 seconds, 12 frames/minute, 720 frames/hour, 3600 frames/5 hours (max game)
- 1 frame every second, 60 frames/minute, 3600 frames/hour, 18000 frames/5 hours
- run batch python program to extract PGN file from video
Android app
I google "android video capture a image every 10 seconds" and find
few
APPs to do
TimeLapse, as
- Framelapse
- Lapse It
- Tina Time-lapse : de 1 fps a 50 fps, hasta 4K.
You simply set the photo interval and the image resolution, and you're good to go.
The capture screen has just three buttons for start, pause and stop, and shows a small status bar too.
- AutoPIX - recodrs GPS location also
Look for them in Play Store
App - Lapse It
Description:
- 2 £ - free version available
- export to H264, MP4, MOV and FLV
- frame interval : 1 .. 99999
- time scale : milliseconds, seconds, minutes
- resolution : 360p .. 1080p
- output folder : /storage/C871-3D8E/Android/data/com.ui.Lapseit/files/Lapseit
- encoder : MP4, H264, MOV
Features not available in free app:
- use both phone cameras
- high definition
- enhancing effects
- resume capture
- sound tracks
- no ads
- updates
Storage use calculation :
- 7 frames at 480p using 0,34 MB
webcam python code
C:\sebas\miscosas\albert\python\webcam> type veure.py
import cv2
print "### CV2 version I have is " + cv2.__version__
import sys
print "### SYS version I have is " + str(sys.version_info[0]) + "." + str(sys.version_info[1]) + "." + str(sys.version_info[2])
# 0 = primera webcam
cap = cv2.VideoCapture(0)
if not cap:
print '--- no trobo la webcam'
sys.exit(1)
while( cap.isOpened() ):
# Capture frame-by-frame
ret, frame = cap.read()
# Our operations on the frame come here
imggray = cv2.cvtColor( frame, cv2.COLOR_BGR2GRAY )
# Display the resulting frame
if ret:
# print dir ( frame ) # display all object fields
cv2.imshow( 'my webcam window', imggray )
cv2.waitKey( 50 )
else:
print '--- el frame no va be'
if cv2.waitKey(50) & 0xFF == ord('q'):
print '>>> adeu'
break
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
FER.CMD has "python veure.py" and says :
import cv2
ImportError: No module named cv2
C:\python27\Lib\site-packages> copy C:\opencv\build\python\2.7\x64\cv2.pyd
import cv2
ImportError: DLL load failed: %1 is not a valid Win32 application.
C:\python27\Lib\site-packages> copy C:\opencv\build\python\2.7\x86\cv2.pyd
import cv2
ImportError: numpy.core.multiarray failed to import
C:\python27\Lib\site-packages> python -m pip install --upgrade numpy
Collecting numpy
Downloading numpy-1.11.0-cp27-none-win32.whl (6.5MB)
100% |################################| 6.5MB 36kB/s
Installing collected packages: numpy
Successfully installed numpy-1.11.0
webcam input files
- tauler B/N, android a la ma, 10 jugades
- tauler marró-marró, android a la ma, 10 jugades
webcam project links
motion detection project
Volem generar una alarma en detectar moviment en un video
video recording with motion detection, in Python
RaspberryPi home surveillance with only ~150 lines of Python code -
github -> uses
Flask - Flask is a micro web framework written in Python
openCV links
Write into BLOG
Blogger Data API allows client applications to view and update Blogger content -
reference
Go to
Blogger
sag :
proves python
W500, 20170126
C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests> run_all_tests.py
Traceback (most recent call last):
File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests\run_all_tests.py", line 26, in <module>
import run_data_tests
File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests\run_data_tests.py", line 9, in <module>
import gdata_test
File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests\gdata_test.py", line 26, in <module>
import gdata
ImportError: No module named gdata
Improve PYTHONPATH :
C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests> echo %PYTHONPATH%
C:\opencv\build\python\2.7\x86\;C:/sebas/miscosas/python/client-code/gdata-python-client-master/src;
Now we have :
C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests> run_all_tests.py
Traceback (most recent call last):
File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests\run_all_tests.py", line 26, in <module>
import run_data_tests
File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests\run_data_tests.py", line 19, in <module>
import gdata_tests.auth_test
File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests\gdata_tests\auth_test.py", line 24, in <module>
import gdata.auth
File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\src\gdata\auth.py", line 29, in <module>
import gdata.oauth.rsa as oauth_rsa
File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\src\gdata\oauth\rsa.py", line 10, in <module>
from tlslite.utils import keyfactory
ImportError: No module named tlslite.utils
Get
tls lite plus "python setup.py install" :
error: Unable to find vcvarsall.bat
Raspberry
We have (20170127)
pi@odin:~/blog $ python -V
Python 2.7.9
- download the client library
- download and install the ElementTree package
Verify the package is already installed with python :
pi@odin:~/blog $ python
Python 2.7.9 (default, Mar 8 2015, 00:52:26)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from xml.etree import ElementTree
>>>
Download :
pi@odin:~/blog/client $ git clone https://github.com/google/gdata-python-client.git
Cloning into 'gdata-python-client'...
remote: Counting objects: 10326, done.
remote: Total 10326 (delta 0), reused 0 (delta 0), pack-reused 10326
Receiving objects: 100% (10326/10326), 24.60 MiB | 2.27 MiB/s, done.
Resolving deltas: 100% (7567/7567), done.
Checking connectivity... done.
pi@odin:~/blog/client $
Install :
pi@odin:~/blog/client/gdata-python-client $ sudo python ./setup.py install
Run tests :
pi@odin:~/blog/client/gdata-python-client/tests $ ./run_data_tests.py
Traceback (most recent call last):
File "./run_data_tests.py", line 19, in <module>
import gdata_tests.auth_test
File "/home/pi/blog/client/gdata-python-client/tests/gdata_tests/auth_test.py", line 24, in <module>
import gdata.auth
File "/usr/local/lib/python2.7/dist-packages/gdata/auth.py", line 29, in <module>
import gdata.oauth.rsa as oauth_rsa
File "/usr/local/lib/python2.7/dist-packages/gdata/oauth/rsa.py", line 10, in <module>
from tlslite.utils import keyfactory
ImportError: No module named tlslite.utils
Get it
pi@odin:~/blog/tls-lite $ wget http://trevp.net/tlslite/tlslite-0.3.8.tar.gz
--2017-01-27 10:08:03-- http://trevp.net/tlslite/tlslite-0.3.8.tar.gz
Resolving trevp.net (trevp.net)... 204.11.246.1
Connecting to trevp.net (trevp.net)|204.11.246.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 637558 (623K) [application/x-gzip]
Saving to: âtlslite-0.3.8.tar.gzâ
tlslite-0.3.8.tar.gz 100%[=========================>] 622.62K 191KB/s in 3.3s
2017-01-27 10:08:07 (191 KB/s) - âtlslite-0.3.8.tar.gzâ saved [637558/637558]
Uncompress :
pi@odin:~/blog/tls-lite $ tar -xvf tlslite-0.3.8.tar.gz
Install :
pi@odin:~/blog/tls-lite/tlslite-0.3.8 $ ./setup.py install
Still not seen - try again :
pi@odin:~/blog/client/gdata-python-client/tests $ sudo pip install --upgrade tlslite
Downloading/unpacking tlslite
Downloading tlslite-0.4.9.tar.gz (105kB): 105kB downloaded
Running setup.py (path:/tmp/pip-build-89_5kn/tlslite/setup.py) egg_info for package tlslite
warning: no files found matching '*' under directory 'docs'
Installing collected packages: tlslite
Running setup.py install for tlslite
changing mode of build/scripts-2.7/tls.py from 644 to 755
changing mode of build/scripts-2.7/tlsdb.py from 644 to 755
warning: no files found matching '*' under directory 'docs'
changing mode of /usr/local/bin/tls.py to 755
changing mode of /usr/local/bin/tlsdb.py to 755
Successfully installed tlslite
Cleaning up...
pi@odin:~/blog/client/gdata-python-client/tests $
Now we have :
pi@odin:~/blog/client/gdata-python-client/tests $ ./run_data_tests.py
Traceback (most recent call last):
. . .
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1514, in _get
with open(path, 'rb') as stream:
IOError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/protobuf-3.1.0.post1-py2.7.egg/EGG-INFO/namespace_packages.txt'
So we have permission problems :
pi@odin:~/blog/client/gdata-python-client/tests $ sudo ./run_data_tests.py
. . . {very large}
test_generate_timestamp (gdata_tests.oauth.data_test.OauthUtilsTest) ... ok
----------------------------------------------------------------------
Ran 39 tests in 0.032s
OK
Running the sample code
A full working sample client, containing all the sample code shown in this document, is available in the BloggerExample.py file,
under the directory gdata-python-client/samples/blogger/.
You can run the sample with the following arguments:
python BloggerExample.py --email [email_address] --password [password]
El resultat es :
pi@odin:~/blog/client/gdata-python-client/samples/blogger $ python BloggerExample.py --email sebastiasebas@gmail.com --password mypwd
Please choose the authorization mechanism you want to use.
1. [deprecated] to use your email address and password (ClientLogin)
2. [deprecated] to use a web browser to visit an auth web page (AuthSub)
3. [deprecated] if you have registed to use OAuth
4. if you have registed to use OAuth 2.0
: 1
Traceback (most recent call last):
. . .
File "/usr/local/lib/python2.7/dist-packages/gdata/client.py", line 373, in request_client_login_token
response, ClientLoginFailed, response_body)
gdata.client.ClientLoginFailed: Server responded to ClientLogin request: 404, https://developers.google.com/accounts/docs/AuthForInstalledApps
Googl'ing gets
Important: ClientLogin has been officially deprecated since April 20, 2012 and is now no longer available.
Requests to ClientLogin will fail with a HTTP 404 response.
We encourage you to migrate to OAuth 2.0 as soon as possible.
Que hem de fer ara ? Anem pel OAuth !
Explicació :
- Obtain OAuth 2.0 credentials from the Google API Console {seguir "link" a "explicacio"}
Visit the Google API Console to obtain OAuth 2.0 credentials such as a client ID and client secret that are known to both Google and your application.
- Obtain an access token from the Google Authorization Server.
- Send the access token to an API.
- Refresh the access token, if necessary.
Tot va be :
4. if you have registed to use OAuth 2.0
: 4
Please enter your OAuth 2.0 Client ID which identifies your app: 51601928208-6qo669nb5u3rv1d7fbotvpt1givuls8u.apps.googleusercontent.c
Please enter your OAuth 2.0 Client secret which you share with the OAuth provider: <secret number> + CR
Visit the following URL in your browser to authorize this app:
https://accounts.google.com/o/oauth2/auth?access_type=offline&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&client_id=5160
What is the verification code? 4/muC2rg5OkbtJmEx5vTIno9GI_DpeLaXvfvt9Ii01BMU
ComGiraElMon's Blogs
proves de python
Successfully created draft post: "Snorkling in Aruba".
Successfully created public post: "Back from vacation".
Now listing all posts.
proves de python
Back from vacation
No Title
primera entrada
Now listing all posts between 2007-04-04 and 2007-04-23.
2007-04-04
updated
proves de python posts between 2007-04-04 and 2007-04-23
proves de python
Now updating the title of the post we just created:
Successfully changed the post's title to "The party's over".
Now posting a comment on the post titled: "The party's over".
Successfully posted "Did you see any sharks?" on the post titled: "The party's over".
Now printing all comments
Comments on proves de python: The party's over
Did you see any sharks?
2017-04-25T03:31:38.593-07:00
Now deleting the comment we just posted
Successfully deleted comment.
Comments on proves de python: The party's over
Now deleting the post titled: "The party's over".
Successfully deleted post.
proves de python
No Title
primera entrada
pi@odin:~/blog/client/gdata-python-client/samples/blogger $
Request a Google login()
Minie code : /home/sebas/dades/python/google_auth
- visit https://console.cloud.google.com/ and log in with your Google account (benet)
- click "Select a project" and either choose an existing project or create a new one.
just request login
- go to "APIs & Services" > "Library" (sota les 3 ratlles horitzontals de dalt a l'esquerra)
search for the Google APIs your app needs (e.g., Gmail API, Drive API) and click "Enable" : Drive and Mail
- navigate to "APIs & Services" > "OAuth consent screen"
- App name := just_login
- eMail := benet.papell@gmail.com
- Audience := internal
- go to "APIs & Services" > "Credentials" and click "Create Credentials" > "OAuth client ID"
App type := desktop ; Name := "python_code"
- click "Download JSON" :
client_secret_530674411431-erbl6f8olnffacldaldco0hodf0m8vck.apps.googleusercontent.com.json
Here is the result :
sebas@minie:~/dades/python/google_auth$ ./3_gemini.py
--- Credential Information ---
User ID: 107422452309232518508
Email: campdefabes@gmail.com
Access Token (DO NOT SHARE): yb29.a0ATi...
You are authenticated and ready to make API calls!
Sounds
Where to get them :
omxplayer
Documentation :
sag@odin:~/sounds $ omxplayer -o local buzzer.wav
* failed to open vchiq instance
sag@odin:~/sounds $ sudo omxplayer -o local buzzer.wav
[sudo] password for sag:
Audio codec pcm_s16le channels 2 samplerate 44100 bitspersample 16
Subtitle count: 0, state: off, index: 1, delay: 0
have a nice day ;)
aplay
sag@odin:~/sounds $ aplay buzzer.wav
xcb_connection_has_error() returned true
sag@odin:~/sounds $ sudo aplay buzzer.wav
PuTTY X11 proxy: wrong authentication protocol attempted xcb_connection_has_error() returned true
Playing WAVE 'buzzer.wav' : Signed 16 bit Little Endian, Rate 44100 Hz, Stereo
Si no funciona, pot ser
url :
sag@odin:~ $ sudo amixer controls
PuTTY X11 proxy: wrong authentication protocol attempted
numid=3,iface=MIXER,name='PCM Playback Route'
numid=2,iface=MIXER,name='PCM Playback Switch'
numid=1,iface=MIXER,name='PCM Playback Volume'
numid=5,iface=PCM,name='IEC958 Playback Con Mask'
numid=4,iface=PCM,name='IEC958 Playback Default'
sag@odin:~ $
Tambe pot ser
sag@odin:~ $ aplay -l
aplay: device_list:268: no soundcards found...
pyglet
pyglet makes it easy to play and mix multiple sounds together in your game.
The following example plays an MP3 file :
- sudo apt update
- sudo apt install pyglet : "E: Unable to locate package pyglet"
- sudo pip install pyglet : "Successfully installed pyglet future"
import pyglet
music = pyglet.resource.media('music.mp3')
music.play()
pyglet.app.run()
As with the image loading example presented earlier,
pyglet.resource.media locates the sound file in the application's directory (not the working directory).
If you know the actual filesystem path (either relative or absolute), use pyglet.media.load :
import pyglet
explosion = pyglet.media.load('explosion.wav', streaming=False)
explosion.play()
xcb_connection_has_error() returned true
Short sounds, such as a gunfire shot used in a game, should be decoded in memory before they are used,
so that they play more immediately and incur less of a CPU performance penalty.
Specify "streaming=False" in this case:
import pyglet
sound = pyglet.resource.media( 'shot.wav', streaming=False )
sound.play()
url
This
example
demonstrates playing many short audio samples simultaneously, as in a game.
pygame
import pygame
pygame.mixer.init()
pygame.mixer.music.load("abc.wav")
pygame.music.mixer.play()
while pygame.mixer.music.get_busy == True"
continue
url with "pygame.mixer.init()" details
python launcher icon
- find existing launcher's icon location :
- right-click on desktop launcher
- select "Open with Text Editor"
find something like "Icon=cinnamon-panel-launcher"
- use "Nemo" to locate that file -> "/usr/share/icons/hicolor/scalable/apps"
- use it on a new launcher
Fem servir IA per escriure codi de python
Per llegir PDFs, al Minie fem servir PDFPLUMBER
Per generar ODSs, la llibreria pandas (amb el motor odfpy)
Aqui :
ls /home/sebas/dades/ai/*
4 drwxrwxr-x 2 sebas sebas 4096 Jan 2 19:48 bakup_Rx_a_USB
4 drwxrwxr-x 2 sebas sebas 4096 Jan 13 12:55 bolonyesa
4 drwxrwxr-x 5 sebas sebas 4096 Jan 19 20:34 cares
4 drwxrwxr-x 2 sebas sebas 4096 Nov 6 14:23 diskuse
4 drwxrwxr-x 2 sebas sebas 4096 Jan 8 17:17 factures_bonpreu
4 drwxrwxr-x 3 sebas sebas 4096 Jan 10 17:34 films_quality
4 drwxrwxr-x 2 sebas sebas 4096 Jan 13 20:08 html_taula
4 drwxrwxr-x 2 sebas sebas 4096 Jan 7 12:38 spreadsheet
4 drwxrwxr-x 4 sebas sebas 4096 Feb 3 16:59 thermometre
4 drwxrwxr-x 2 sebas sebas 4096 Nov 6 12:23 ukraine_map
4 drwxrwxr-x 2 sebas sebas 4096 Nov 6 11:23 usa_map
4 drwxrwxr-x 2 sebas sebas 4096 Nov 17 15:13 ww2_map
quina es la millor IA per escriure codi de python
Escollir la "millor" IA per escriure codi Python depèn molt de les teves necessitats específiques, el teu flux de treball i el teu pressupost.
Tanmateix, hi ha diversos líders al mercat que són molt valorats per les seves capacitats:
Opció més popular i versàtil:
- GitHub Copilot
És extremadament popular i àmpliament adoptat.
S'integra perfectament amb els IDEs més utilitzats (VS Code, JetBrains) i ofereix suggeriments de codi en temps real,
complecions intel·ligents i fins i tot ajuda amb la generació de proves i documentació.
Està impulsat pel model OpenAI Codex (i ara també per GPT-4o i altres models més nous), el que el fa molt potent per generar codi rellevant i contextualitzat.
Altres opcions destacades amb característiques fortes:
- Tabnine
Conegut per les seves suggeriments de codi sensibles al context i la seva flexibilitat.
S'adapta al teu estil de codificació i pot funcionar amb models al núvol o locals, cosa que és important per a la privadesa del codi.
És compatible amb molts IDEs i llenguatges.
- Amazon CodeWhisperer
Si treballes molt amb AWS, CodeWhisperer és una opció excel·lent, ja que s'integra sense problemes
amb els serveis d'AWS i està optimitzat per a desenvolupadors de Python en entorns cloud.
Ofereix una versió gratuïta per a desenvolupadors individuals.
- Bito
Aquesta eina es presenta com a molt versàtil i potent per a desenvolupadors Python.
Ofereix assistència de codificació directament en IDEs populars i inclou funcions com la revisió de codi, l'explicació de codi complex i l'optimització del flux de treball.
- Cursor
És un editor de codi basat en VS Code amb capacitats d'IA integrades.
Permet interactuar amb l'IA mitjançant xat, generar codi, depurar i fins i tot fer canvis al codi mitjançant llenguatge natural.
- Aider
Una eina de programació en parella amb IA que funciona des del terminal.
És compatible amb diversos LLMs (locals i al núvol), mapeja la teva base de codi completa per a un context profund,
s'integra amb Git i pot fins i tot fer correccions automàtiques basades en linters i tests.
Si la ciència de dades és el teu focus:
una altra opinió
Here are the top 3 AI assistants for Python coding:
- 1. Cursor - best overall with deep codebase integration and AI-first editor
- 2. GitHub Copilot - most popular, seamless IDE integration with excellent completions
- 3. Claude - strongest for complex logic and understanding nuanced requirements
Bonus: ChatGPT-4 remains excellent for explaining concepts and debugging assistance.
For pure Python work, I'd recommend starting with Cursor as it's specifically designed for AI-powered coding.
cursor ai
homepage
This site presents Cursor as a cutting-edge AI-powered coding tool
designed to make software development more productive
by integrating advanced AI models and providing deep codebase understanding.
copilot ai
homepage
an AI companion designed to inform, entertain, inspire, provide advice, feedback, and straightforward answers.
claude ai
homepage
next-generation AI assistant, focused on being safe, accurate, and secure to help users with various tasks including writing, coding, and collaboration.
"prompts" que he fet servir
- fitxer HTML : posar "timestamp" al HTML i a la consola
comença per posar una marca temporal a "console.log()" amb un texte com "*** comencem ***"
fes servir "console.log()" en cada una de les branques en que prenguis decisions - vull un log ben ple
has de marcar el titol de la pagina HTML amb una marca temporal per poder diferenciar les versions que anem generant
posa-hi també el nom del programa python que la ha generat
- fitxer HTML : posar sempre les meves capçaleres
<BASE TARGET="_top">
<meta name="robots" content="noindex,nofollow">
<meta charset="utf-8">
<LINK REL="SHORTCUT ICON" HREF="./icons/favicon.ico">
<LINK REL=STYLESHEET HREF="./sag.css" TYPE="text/css">
- programa python : fer sempre un log
{program.log} must contain a indication of all the branches in the execution,
together with the data associated with it
at the end of the execution, we must have
*) initial and final timestamp
*) input filesnames (input data) and output filenames (output data and log)
- programa python : agafar nom del fitxer "*.py" per generar "*.log"
i need a python program, using python3
start the code with the directive "#!/usr/bin/env python3" line,
followed by the line "# -*- coding: UTF-8 -*-"
next, insert a comment line containing a timestamp into the code
first of all, the program has to retrive its own name (similar to "program.py") and create a filename where it shall log all actions.
if program name was "program.py", the log filename shall be "program.log"
start by writing a timestamp in it and remember to close the log file at program end
also write there the parameters provided to the execution
- programa python : posar sempre a la capçalera
#!/usr/bin/python3
# -*- coding: utf-8 -*-
- posa sempre aquest final per fer un modul reutilitzable :
if __name__ == "__main__": # fem un modul reutilitzable
print( "python version", platform.python_version() )
rc = my_code( sys.argv )
sys.exit()
- si ens falta un modul, mai intentis instalar-lo - avisa i acaba
resultats
sebas@minie:~/dades/ai/thermometre/dibuixar$ ./5_dades_n_fitxers_i_png.py
Dubtes python
- how to set "__version__" ?
It is a dunder
- python what goes in "__init__.py" ?
- qui actualitza el directori "__pycache__" que hi ha sota "__init__.py"
- what's the difference between a module and a package ?
- when do we use "var" to define a variable ?
- PYTHONUNBUFFERED
Setting PYTHONUNBUFFERED=TRUE or PYTHONUNBUFFERED=1 (they are equivalent) allows for log messages to be immediately dumped to the stream instead of being buffered.
- do we need PYTHONPATH ?
The PYTHONPATH envir var is used to locate and import other modules, as openCV
- what is NUMPY ?
wiki
NumPy is the fundamental package for scientific computing with python.
- how to verify "numpy" is working fine ?
After
installation, open python IDLE.
Enter import numpy and display its version
- getting the MAC from python is quite tricky ...
stackoverflow
Links
- Documentation,
language
Reference,
standard
Library
- Intro to python
- baix i creixent
- Python Data Science Handbook { dur pero complert }
- el classic
- hard way
- altres llibreries per python - el pip es molt popular per aixo, una mena de repositori bestial de coses diverses.
PyPI = Python Package Index (PyPI) is a repository of software for python
Per exemple python requests (que per cert, és el millor per fer tcp/ip)
- python for Android
- python tutorials at guru99
- interesting code :
- pythonic argument parser
- python networking
- good site : tutorial,
pop3 client,
socket programming, ...
- series of articles to demonstrate how to use the modules of the python 3 standard library {***}
- lots of tutorials ,
lots of samples (as random number generator)
- random number generation
- use APIs to send tomorrow's weather by email -
see github code,
open weather map api ,
gmail api
- 25 python project ideas
- use pylint to improve coding style
- bytearray objects
- version 3.9
whats new in python 3.9
- call nmap from python
- how to interact with APIs in python
- geeks : response JSON python requests - molt complert
- jQ : jq is a Linux command line utility that is easily used to extract data from JSON documents
- documenting your python code base using docstrings {#}
Docstring conventions are described within PEP 257
- syntax checkers :
- Sorting a python dictionary: values, keys, and more
- module index - v 3.10.6
- Learn Python The Hard Way
-
Structuring Your Project from "The Hitchhiker's guide to python"
- online IDE's :
- SCAPY - interactive packet manipulation library written in Python.
- argparse - parser for command-line options, arguments and sub-commands {****}
- llibres :
- Collab - allows you to write and execute Python in your browser
- python tutor - online compiler, AI tutor, and Visual Debugger for Python, Java, C, C++, and JavaScript