home / infca / python (navigation links) De la mà d'en Albert hem arribat a aquest nou mon

intro, venv, variables, modules and packages, modules, own module + how it is found : *** global access to my modules ***
install, pip, go v3 and update alternatives (select 3.6 or 3.7), | rrdtool problem with python3
header, display version runtime, parametres, arguments, quotes, procedures | exceptions | logging | timestamp
if .. then .. else , while loop | print()
data types : lists, tuples, sets, dictionaries
r/w files | parse | log 2 html | PEP8 | debug
calling python : from bash, from node, .... from python ... | numpy & pandas
TCP/IP, ping, FTP | opening URLs : urllib2 or requests ; pvoutput requests, by CURL or python | REST API server | VPN
CSV | using JSON | matplotlib
*** els meus projectes *** | poplib - imaplib | openCV - ubuntu | blog | request gLogin() | sounds | gpio | web server
Google Sheets : mine or by chatGPT | gspread
Google Custom Search : mine
IA/AI : *** my prompts *** | tricks | dubtes | Links | End | mapa

My first steps into python

go 2 top
Introduction

Python is an interpreted, high-level, generale programming language

Python has a design philosophy that emphasizes code readability, notably using significant whitespace {wiki}

Python is a strongly typed language

Code sample :

sag@odin:~/gpio_python $ cat bucle_sortida_pin37.py #!/usr/bin/python3 # -*- coding: utf-8 -*- # executem amb "sudo ./bucle_sortida_pin37.py" __author__ = "Ivan Grokhotkov" __version__ = "3.2015.2" import sys if sys.version_info[0] == 3: # espressif from urllib.request import urlretrieve from urllib.request import urlopen unicode = lambda s: str(s) # noqa: E731 else: # not Python 3 - today, it is most likely to be Python 2 from urllib import urlretrieve from urllib import urlopen # === def my_code( argv ): ''' Large description ''' while True: try: some_code except KeyboardInterrupt: print( "--- kbd Ctrl-C : bye" ) break # exit the "forever" loop return 0 # if __name__ == "__main__": # fem un modul reutilitzable print( "python version", platform.python_version() ) rc = my_code( sys.argv ) sys.exit()

#!/usr/bin/env python3 # -*- coding: utf-8 -*- """ NOM DEL PROJECTE: [Nom del fitxer] DESCRIPCIÓ: [Què fa el programa] AUTOR: [Nom de l'Usuari / Gemini] DATA: 21 de febrer de 2026 VERSIÓ: 1.2 PROMPT ORIGINAL: # "[Aquí copiaré el teu prompt]" """ import sys import platform import os from datetime import datetime def my_code(args: list[str]) -> int: # 1. Identificació del programa i fitxer de log program_name = os.path.basename(args[0]) log_filename = os.path.splitext(program_name)[0] + ".log" # 2. Gestió del Log amb context manager with open(log_filename, "a", encoding="utf-8") as log_file: # Inici: Timestamp i Paràmetres start_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S") log_file.write(f"[{start_time}] --- INICI EXECUCIÓ ---\n") log_file.write(f"[{start_time}] Paràmetres: {args[1:]}\n") # --- ESPAI PER A LA LÒGICA PRINCIPAL --- # (Aquí anirà el codi que em demanis) log_file.write(f"[{datetime.now().strftime('%H:%M:%S')}] Executant accions...\n") # Final: Timestamp de tancament end_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S") log_file.write(f"[{end_time}] --- FINAL EXECUCIÓ ---\n\n") return 0 if __name__ == "__main__": # fem un mòdul reutilitzable print( "python version", platform.python_version() ) rc = my_code( sys.argv ) sys.exit(rc)

Consells :

The Zen of Python

sebas@minie:~$ python3 Python 3.10.12 (main, Nov 6 2024, 20:22:13) [GCC 11.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import this The Zen of Python, by Tim Peters Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those! >>>

WhirlWinTour

Luís dixit

En mi opinión es un lenguaje muy elegante. Para hacer programas sencillos es muy agradable de usar. Pero tiene un problema serio de portabilidad, por dos razones:

Para arreglar este problema hay docenas de workarounds que requieren que el developer tenga que aprender la gestión de "entornos aislados" y cosas por el estilo. Esto es asumible para un developer voluntarioso, pero si quieres que tu código pueda ser usado por otros, les vas a obligar a tener que aprender a usar esos mecanismos de workaround para que también les funcione a ellos.

venv in python

It is highly recommended that you use python virtual environments, or venvs.

A venv is an isolated copy of the python executable and associated files, which provides a separate environment for installing python modules, independent from the system python installation. You can have as many venvs as you need, which means that you can have multiple module configurations set up, and you can switch between them easily.

Read realpython intro

sebas@minie:~/dades/Cifras_y_Letras$ python3 -m venv env && source ./env/bin/activate (env) sebas@minie:~/dades/Cifras_y_Letras$ python3 -m pip install -U pip wheel setuptools wget https://files.pythonhosted.org/packages/04/8a/d336e6f73bd988630488c3d6b9eb689c360f53ea0751893156ba77591e52/countdown_numbers_solver-1.0.0-py3-none-any.whl (env) sebas@minie:~/dades/Cifras_y_Letras$ pip install countdown_numbers_solver-1.0.0-py3-none-any.whl Processing ./countdown_numbers_solver-1.0.0-py3-none-any.whl Installing collected packages: countdown-numbers-solver Successfully installed countdown-numbers-solver-1.0.0 (env) sebas@minie:~/dades/Cifras_y_Letras$ env) sebas@minie:~/dades/Cifras_y_Letras$ python Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import countdown_numbers_solver >>> countdown_numbers_solver.solve([ 100, 25, 8, 3, 1, 1 ], 984) ( 100 + 25 + 1 - 3 ) x 8 ( 100 + 25 - 1 - 1 ) x 8 ( 100 + 1 ) x ( 8 + 1 ) + 25 x 3 ( ( 100 - 1 ) / 3 + 8 ) x ( 25 - 1 ) Total: 4 solutions. >>> >>> countdown_numbers_solver.solve([ 1, 2, 3, 4, 5, 6 ], 279) ( 6 x 5 + 1 ) x ( 4 + 3 + 2 ) ( 6 x ( 3 + 2 ) + 1 ) x ( 5 + 4 ) ( 6 + 2 ) x ( 4 + 3 ) x 5 - 1 ( ( 4 + 2 ) x 5 + 1 ) x ( 6 + 3 ) Total: 4 solutions. >>> countdown_numbers_solver.solve([ 7, 4, 4, 9, 9, 2 ], 958) Total: 0 solutions. (venv) $ deactivate $

Another project :

  1. create a new directory for your project and navigate into it.
  2. set up a virtual environment for Python dependencies:

    python3 -m venv venv source venv/bin/activate
  3. install Flask and Flask components :

    pip install Flask Flask-SQLAlchemy Flask-Login flask-wtf email_validator

Flask


python variables

In many programming languages, variables are best thought of as containers or buckets into which you put data.

In Python, by contrast, variables are best thought of not as containers but as pointers.
one consequence of this: because Python variables just point to various objects, there is no need to "declare" the variable, or even require the variable to always point to information of the same type!

x = 1 # x is an integer x = 'hello' # now x is a string x = [1, 2, 3] # now x is a list

There is a consequence of this "variable as pointer" approach that you need to be aware of. If we have two variable names pointing to the same mutable object, then changing one will change the other as well!

WhirlWindTour ch.03

python types - Built-In Data Structures

Python's simple types are summarized in the following table:

** Python Scalar Types ** Type Example Description ----- -------- ------------ int x = 1 integers (i.e., whole numbers) float x = 1.0 floating-point numbers (i.e., real numbers) complex x = 1 + 2j Complex numbers (i.e., numbers with real and imaginary part) bool x = True Boolean: True/False values str x = 'abc' String: characters or text NoneType x = None Special object indicating nulls

Python also has several built-in compound types, which act as containers for other types. These compound types are:

** Python Compound Types ** Type Example Description ----- -------- ------------ list [1, 2, 3] Ordered collection tuple (1, 2, 3) Immutable ordered collection dict {'a':1, 'b':2, 'c':3} Unordered (key,value) mapping set {1, 2, 3} Unordered collection of unique values

WhirlWindTour ch.06


python modules and packages

Modules and packages are two mechanisms that facilitate modular programming.

Any python file is a module, its name being the file's base name without the .py extension.
A package is a collection of python modules: while a module is a single python file, a package is a directory of python modules containing an additional __init__.py file.

stack overflow

A package or module name can *not* start with a number - stackoverflow - it will trigger a SyntaxError

There are three different ways to define a module in python:

real python modules and packages {***}

A module is a simple python file that contains collections of functions and global variables and with having a .py extension file.

A package is a directory having python modules and also having a __init__.py file.

geeks 4 geeks

"dir(module)" returns the names of the attributes of the "module" (list of methods, properties and submodules)

dir() does not list the names of built-in functions and variables.
If you want a list of those, they are defined in the standard module builtins:

>>> import builtins >>> dir(builtins)

For all the details of a module, try "help(module_name)"

python modules

A module is a file containing python definitions and statements. The file name is the module name with the suffix .py appended. The definitions from a module can be imported into other modules or into the main module.

import statement

The statement

import fibo

... does not enter the names of the functions in the current symbol table ; it only enters the module name.

Using the module name you can access the functions :

>>> fibo.fib2(1000)

If you intend to use a function often you can assign it a local name :

>>> myfib = fibo.fib2 >>> myfib(1000)

There is a variant of the import statement that imports names from a module directly into the importing module’s symbol table.

>>> from fibo import fib2, fib3 >>> fib3(500)

This does not introduce the module name from which the imports are taken in the local symbol table, so in the example, fibo is not defined.

If the module name is followed by as, then the name following as is bound directly to the imported module.

>>> import fibo as fibi >>> fibi.fib2(500)

This is effectively importing the module in the same way that import fibo will do, with the only difference of it being available as fibi.

It can also be used when using from with similar effects:

>>> from fibo import fib2 as fibonacci >>> fibonacci(500)

The important one for me is :

sebas@pi0alby:~/python/bot $ cat veure_versio.py #!/usr/bin/python3 from telebot import version as my_versio_tb print( "(0) tb version", my_versio_tb.__version__ )
The Module Search Path

Let’s take a look at what happens when python executes the statement:

import mod # realpython

When the interpreter executes the above import statement, it searches for mod.py in a list of directories assembled from the following sources:

The resulting search path is accessible in the Python variable sys.path, which is obtained from a module named sys:

>>> import sys >>> sys.path ['', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/usr/local/lib/python3.10/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.10/dist-packages'] >>>
good "import" sample

Goal : we want to have some config values in a file, included in .gitignore

  1. in main progarm "bot.py", we code

    import config # config.py holds bot token as "bot_token"

  2. we specify the configuration values :

    nicolau@mars:~/sebas/python/telegram/telegram.ext/smokey$ cat config.py bot_token='1234567890:AABBCCDDEEffggxQYV3wQqDLL8zNfrJwPot'

  3. finally we use them in main code (bot.py)

    nicolau@mars:~/sebas/python/telegram/telegram.ext/smokey$ cat bot.py updater = Updater( config.bot_token, workers=10, use_context=True )
own project with own import file

Al fitxer arrel, posem :

pi@R4:~/python/pkw/bin $ cat pkw_cli.py import pkw_pkg

I al directori hi tenim :

pi@R4:~/python/pkw/bin $ ls pkw_pkg/ 4 -rw-r--r-- 1 pi pi 118 May 7 22:23 __init__.py 4 -rw-r--r-- 1 pi pi 1614 May 13 21:44 pkw_graph.py 4 -rw-r--r-- 1 pi pi 1525 May 16 22:01 pkw_initialize.py 4 -rw-r--r-- 1 pi pi 3034 May 16 22:03 pkw_reader.py

Ara al fitxer principal podem fer servir les funcions implementades :

pkw_pkg.pkw_initialize.create_rrd(rrd_filename, overwrite=True) t, h, p = pkw_pkg.pkw_reader.defronius(simulate=simulate) pkw_pkg.pkw_graph.fer_png_3_valors ( rrd_filename,
*** how and where to define own package in own module ***

... a package is a directory of python modules containing an additional __init__.py file (even empty)

sebas@minie:~/dades/python/pkw/eines$ python3 Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux >>> from newp import bubu # from package import module >>> bubu.vfy() # call module.function (including parenthesis !) *** verificar file fn.txt *** True >>> print(bubu.s) # display string defined in module un string definit a BUBU sebas@minie:~/dades/python/pkw/eines$ ls newp sebas@minie:~/dades/python/pkw/eines/newp$ ls bubu.py __init__.py sebas@minie:~/dades/python/pkw/eines/newp$ cat bubu.py #!/usr/bin/python3 # -*- coding: UTF-8 -*- s = 'un string definit a BUBU' def vfy ( fnCSV = 'fn.txt', bVerbose = True ) : bRC = True print ( '*** verificar file {} ***'.format(fnCSV) ) return bRC

real python {***}

trace "import" sentences

I have one py that finds RF24 and another that does not

sebas@pi0alby:~/nrf24l01/nrf24 $ ./1_getting_started.py from RF24 import RF24, RF24_PA_LOW ImportError: No module named RF24 sebas@pi0alby:~/nrf24l01/rf24libs/RF24/examples_linux $ python3 ./getting_started.py Traceback (most recent call last): RuntimeError: can't open device
How to know how python finds a module ?

We could have

ImportError: No module named <module name>

Anem al directori i fem

$ python3 import sys sys.path

As :

pi@R4:~/python $ python3 Python 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/home/pi/.local/lib/python3.7/site-packages', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages'] >>>
Standard package ubication

The package must be one directory bellow your code :

nicolau@mars:~/sebas/python/fronius$ mkdir pkg_fronius nicolau@mars:~/sebas/python/fronius$ touch pkg_fronius/__init__.py nicolau@mars:~/sebas/python/fronius$ python3 Python 3.7.5 (default, Dec 9 2021, 17:04:37) [GCC 8.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import pkg_fronius >>>

Gracies, Albert !

I el seu contingut :

pi@pi0:~/python/fronius/pkg_fronius $ cat from_fronius_json_extract_dict.py #!/usr/bin/python3 # -*- coding: UTF-8 -*- import json # =============== own functions def get_dictionary( szFN ): print( "(pkg) >>> get a dictionary from file {", szFN, "}" ) try: pf = open( szFN, 'r' ) dt = json.load( pf ) # read json from file into dictionary pf.close() my_body = dt['Body'] my_values = my_body['Data']['inverter/1']['Data']['Temperature_Powerstage']['Values'] ... return my_dict

Comprovem importacio :

pi@pi0:~/python/fronius $ python3 Python 3.7.3 (default, Jan 22 2021, 20:04:44) >>> import pkg_fronius >>> print(pkg_fronius) <module 'pkg_fronius' from '/home/pi/python/fronius/pkg_fronius/__init__.py'> >>> print(dir(pkg_fronius)) ['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__'] >>>

Com el fem servir ?

pi@pi0:~/python/fronius $ cat y_read_temperatures_and_plot_their_max.py import pkg_fronius rd = pkg_fronius.from_fronius_json_extract_dict.get_dictionary()
own packages - global access

{20220806} Has de copiar tots els teus moduls al directori de llibreries de python, al subdirectory que mes t'agradi exemple "sebas".
Llavors podras fer un "import sebas" des de qualsevol python.

  1. ens aclarim amb quin python treballem i el seu directori :

    pi@pi0:~ $ python3 Python 3.7.3 (default, Jan 22 2021, 20:04:44) >>> import sys >>> sys.path ['', '/usr/lib/python37.zip', '/usr/lib/python3.7', ...

  2. hi fem un directori per les nostres eines :

    pi@pi0:/usr/lib/python3.7 $ sudo mkdir sebas_eines

  3. fem que sigui un package :

    pi@pi0:/usr/lib/python3.7 $ sudo touch sebas_eines/__init__.py pi@pi0:/usr/lib/python3.7/sebas_eines $ ls 0 -rw-r--r-- 1 root root 0 Aug 6 11:46 __init__.py

  4. fem proves de "import" :

    pi@pi0:~ $ python3 Python 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sebas_eines

  5. codifiquem un servei :

    pi@pi0:/usr/lib/python3.7/sebas_eines $ cat greeting.py def say_hello(name: str) -> str: return f'Hello {name}!'

  6. codifiquem un client :

    pi@pi0:~/python/proves $ cat mestre.py #!/bin/python3 from sebas_eines import greeting rc = greeting.say_hello("Sebas") print( rc )

  7. verifiquem q funciona :

    pi@pi0:~/python/proves $ ./mestre.py Hello Sebas!
Executing modules as scripts

When you run a python module with ( url )

python fibo.py <arguments>

... the code in the module will be executed, just as if you imported it, but with the "__name__" set to "__main__".

That means that by adding this code at the end of your module:

if __name__ == "__main__": import sys fib(int(sys.argv[1]))

... you can make the file usable as a script as well as an importable module, because the code that parses the command line only runs if the module is executed as the “main” file

Again : the __name__ test at the bottom is true only when this file is run as a top-level script (e.g., from a command line, via an icon click, or within the IDLE GUI), not when it is imported.

display module version

There is no "standard" way :

print( platform.python_version() ) # python print( "NumPy version %s" % numpy.__version__ ) # numerical python print( "CV2 version is " + cv2.__version__ ) # cv2 print( "SYS version is " + str(sys.version_info[0]) + "." + str(sys.version_info[1]) + "." + str(sys.version_info[2]) ) $ python -c "import requests; print( requests.__version__ )" $ python -c 'from telebot import version; print( "telebot version is", version.__version__ )' r0 $ python3 -c 'import matplotlib; print(matplotlib.__version__, matplotlib.__file__)'
display module contents - telebot

from here :

sebas@pi0alby:~/python/bot $ python Python 3.7.3 (default, Dec 20 2019, 18:57:59) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import telebot >>> dir(telebot) ['AsyncTeleBot', 'FileHandlerBackend', 'Handler', 'MemoryHandlerBackend', 'TeleBot', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'apihelper', 'console_output_handler', 'formatter', 'handler_backends', 'logger', 'logging', 'print_function', 're', 'six', 'sys', 'threading', 'time', 'types', 'util'] >>> dir(telebot.TeleBot) ['_TeleBot__non_threaded_polling', '_TeleBot__notify_update', '_TeleBot__retrieve_updates', '_TeleBot__skip_updates', '_TeleBot__threaded_polling', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_build_handler_dict', '_exec_task', '_notify_command_handlers', '_notify_next_handlers', '_notify_reply_handlers', '_test_filter', '_test_message_handler', 'add_callback_query_handler', 'add_channel_post_handler', 'add_chosen_inline_handler', 'add_edited_channel_post_handler', 'add_edited_message_handler', 'add_inline_handler', 'add_message_handler', 'add_middleware_handler', 'add_poll_answer_handler', 'add_poll_handler', 'add_pre_checkout_query_handler', 'add_shipping_query_handler', 'add_sticker_to_set', 'answer_callback_query', 'answer_inline_query', 'answer_pre_checkout_query', 'answer_shipping_query', 'callback_query_handler', 'channel_post_handler', 'chosen_inline_handler', 'clear_reply_handlers', 'clear_reply_handlers_by_message_id', 'clear_step_handler', 'clear_step_handler_by_chat_id', 'create_new_sticker_set', 'delete_chat_photo', 'delete_chat_sticker_set', 'delete_message', 'delete_sticker_from_set', 'delete_webhook', 'disable_save_next_step_handlers', 'disable_save_reply_handlers', 'download_file', 'edit_message_caption', 'edit_message_live_location', 'edit_message_media', 'edit_message_reply_markup', 'edit_message_text', 'edited_channel_post_handler', 'edited_message_handler', 'enable_save_next_step_handlers', 'enable_save_reply_handlers', 'export_chat_invite_link', 'forward_message', 'get_chat', 'get_chat_administrators', 'get_chat_member', 'get_chat_members_count', 'get_file', 'get_file_url', 'get_game_high_scores', 'get_me', 'get_sticker_set', 'get_updates', 'get_user_profile_photos', 'get_webhook_info', 'infinity_polling', 'inline_handler', 'kick_chat_member', 'leave_chat', 'load_next_step_handlers', 'load_reply_handlers', 'message_handler', 'middleware_handler', 'pin_chat_message', 'poll_answer_handler', 'poll_handler', 'polling', 'pre_checkout_query_handler', 'process_middlewares', 'process_new_callback_query', 'process_new_channel_posts', 'process_new_chosen_inline_query', 'process_new_edited_channel_posts', 'process_new_edited_messages', 'process_new_inline_query', 'process_new_messages', 'process_new_poll', 'process_new_poll_answer', 'process_new_pre_checkout_query', 'process_new_shipping_query', 'process_new_updates', 'promote_chat_member', 'register_for_reply', 'register_for_reply_by_message_id', 'register_next_step_handler', 'register_next_step_handler_by_chat_id', 'remove_webhook', 'reply_to', 'restrict_chat_member', 'send_animation', 'send_audio', 'send_chat_action', 'send_contact', 'send_dice', 'send_document', 'send_game', 'send_invoice', 'send_location', 'send_media_group', 'send_message', 'send_photo', 'send_poll', 'send_sticker', 'send_venue', 'send_video', 'send_video_note', 'send_voice', 'set_chat_administrator_custom_title', 'set_chat_description', 'set_chat_permissions', 'set_chat_photo', 'set_chat_sticker_set', 'set_chat_title', 'set_game_score', 'set_my_commands', 'set_sticker_position_in_set', 'set_update_listener', 'set_webhook', 'shipping_query_handler', 'stop_bot', 'stop_message_live_location', 'stop_poll', 'stop_polling', 'unban_chat_member', 'unpin_chat_message', 'upload_sticker_file']

A bit better :

#!/usr/bin/python3 import telebot print( telebot.__path__ ) ['/usr/local/lib/python3.7/dist-packages/telebot']
display module contents - requests

Sembla que amb la versio 2.7 podem fer servir requests.exceptions.HTTPError ...

Revisem la documentacio !

pi@R4:~/python/fronius $ python Python 2.7.16 (default, Oct 10 2019, 22:02:15) [GCC 8.3.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import requests >>> print(dir(requests)) ['ConnectTimeout', 'ConnectionError', 'DependencyWarning', 'FileModeWarning', 'HTTPError', 'NullHandler', 'PreparedRequest', 'ReadTimeout', 'Request', 'RequestException', 'RequestsDependencyWarning', 'Response', 'Session', 'Timeout', 'TooManyRedirects', 'URLRequired', '__author__', '__author_email__', '__build__', '__builtins__', '__cake__', '__copyright__', '__description__', '__doc__', '__file__', '__license__', '__name__', '__package__', '__path__', '__title__', '__url__', '__version__', '_check_cryptography', '_internal_utils', 'adapters', 'api', 'auth', 'certs', 'chardet', 'check_compatibility', 'codes', 'compat', 'cookies', 'cryptography_version', 'delete', 'exceptions', 'get', 'head', 'hooks', 'logging', 'models', 'options', 'packages', 'patch', 'post', 'put', 'pyopenssl', 'request', 'session', 'sessions', 'status_codes', 'structures', 'urllib3', 'utils', 'warnings'] >>> print(dir(requests.exceptions)) ['BaseHTTPError', 'ChunkedEncodingError', 'ConnectTimeout', 'ConnectionError', 'ContentDecodingError', 'FileModeWarning', 'HTTPError', 'InvalidHeader', 'InvalidProxyURL', 'InvalidSchema', 'InvalidURL', 'MissingSchema', 'ProxyError', 'ReadTimeout', 'RequestException', 'RequestsDependencyWarning', 'RequestsWarning', 'RetryError', 'SSLError', 'StreamConsumedError', 'Timeout', 'TooManyRedirects', 'URLRequired', 'UnrewindableBodyError', '__builtins__', '__doc__', '__file__', '__name__', '__package__'] sebas@pi0alby:~/python/bot $ python Python 3.7.3 (default, Dec 20 2019, 18:57:59) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import requests >>> print(dir(requests)) ['ConnectTimeout', 'ConnectionError', 'DependencyWarning', 'FileModeWarning', 'HTTPError', 'NullHandler', 'PreparedRequest', 'ReadTimeout', 'Request', 'RequestException', 'RequestsDependencyWarning', 'Response', 'Session', 'Timeout', 'TooManyRedirects', 'URLRequired', '__author__', '__author_email__', '__build__', '__builtins__', '__cached__', '__cake__', '__copyright__', '__description__', '__doc__', '__file__', '__license__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__title__', '__url__', '__version__', '_check_cryptography', '_internal_utils', 'adapters', 'api', 'auth', 'certs', 'chardet', 'check_compatibility', 'codes', 'compat', 'cookies', 'cryptography_version', 'delete', 'exceptions', 'get', 'head', 'hooks', 'logging', 'models', 'options', 'packages', 'patch', 'post', 'put', 'pyopenssl', 'request', 'session', 'sessions', 'status_codes', 'structures', 'urllib3', 'utils', 'warnings'] >>> print(dir(requests.ConnectTimeout)) ['__cause__', '__class__', '__context__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', '__suppress_context__', '__traceback__', '__weakref__', 'args', 'characters_written', 'errno', 'filename', 'filename2', 'strerror', 'with_traceback']
display module version - requests

We can use importlib.metadata since python 3.8 :

print( "(4) version telebot estatica" ) # python >=3.8 from importlib.metadata import version version('telebot')

Yep ! Went into importlib_metadata

python "class" and "self" and "__init__" methods

url

split python code into modules

Lets have "main" file demo.py :

#!/usr/bin/python3 # -*- coding: utf-8 -*- ''' Demo for the TP-Link A19-LB130 RBGW WiFi bulb ''' import sys import time from tplight import LB130 def my_code( argv ): ''' Main program function ''' light = LB130("10.0.0.130") # create an instance of the light with its IP address if __name__ == "__main__": my_code( sys.argv ) sys.exit()

The secondary file is tplight.py :

#!/usr/bin/python3 # -*- coding: utf-8 -*- ''' Control class for TP-Link A19-LB130 RBGW WiFi bulb ''' import datetime import socket import json import sys class LB130(object): ''' Methods for controlling the LB130 bulb ''' encryption_key = 0xAB __udp_ip = "10.0.0.130"
calling python from python without modules

os.system('python calendario.py') subprocess.getoutput('python3 nom.py')

python versions
python v2 (2.6) vs v3 (3.6) main differences

Very few :

python.org docs, url

some more differences I've found

how to format print()

if err: my_node_version = '--- %s ---' % [err] szName = "My name is {}, I'm {}".format( "John", 36 ) >>> v0, v1, v2 = "a", "b", "c" >>> print("%s%s%s" % (v0, v1, v2)) abc >>> print("%s%s%s".format(v0, v1, v2) abc >>> print(f"{v0}{v1}{v2}") abc

From an improved string formatting syntax Guide :

%-formatting - name = "Eric" ; age = 74 ; "Hello, %s. You are %s." % (name, age) f-Strings - f"Hello, {name}. You are {age}." str.format() - "Hello, {}. You are {}.".format(name, age) print( 'Tot {:.2f}'.format (ftot) ) - 2 decimals only

f-strings are expressions evaluated at runtime rather than constant values !

If we want to provide formatting information :

val = 12.3 print(f'{val:.2f}') # 12.30 print(f'{val:.5f}') # 12.30000

zetcode : width, justify, numeric notations,

strings and bytes literals

python 3.x makes a clear distinction between the types:

Bytes literals are always prefixed with 'b' or 'B'; they produce an instance of the bytes type instead of the str type.

You use "str" when you want to represent text.
You use "bytes" when you want to represent low-level binary data like structs.

You can encode a str to a bytes object :

>>> '\uFEFF'.encode('UTF-8') b'\xef\xbb\xbf'

And you can decode a bytes into a str :

>>> b'\xE2\x82\xAC'.decode('UTF-8') '€'

docs.python.org

Unicode literal strings

Python 2 uses two different types for strings, unicode and str.

When you define a literal string the interpreter actually creates a new object of type str that holds this literal.

s = "A literal string" print type(s) <type 'str'>

You can explicitly define a literal string as unicode using the u prefix.

The interpreter will create instead a unicode object for this literal.

s = u"A literal string" print type(s) <type 'unicode'>

For ASCII text, using str type is sufficient but if you intend to manipulate non-ASCII text it is important to use unicode type for character level operations to work correctly.

stackoverflow

Unicode details

docs.python {****}


installing python packages

Some python packages can be found in the Raspbian archives, and can be installed using apt, for example:

sudo apt update sudo apt install python-picamera

Not all python packages are available in the Raspbian archives, and those that are can sometimes be out of date. If you can't find a suitable version in the Raspbian archives, you can install packages from the Python Package Index (known as PyPI).

To do so, install pip:

sudo apt-get install python-pip {python v2} sudo apt-get install python3-pip {python v3}

Then install python packages (e.g. simplejson) with pip3:

pip3 install simplejson pip3 install ifaddr python3 -m pip install ifaddr

It is also easy to remove them :

python3 -m pip install textparser python3 -m pip uninstall textparser

And to list the installed packages :

$ sudo pip3 list $ apt list --installed $ dpkg -l

Sometimes we need to upgrade a package :

$ python -m pip install --upgrade pip

Sometimes, if you dont use "sudo", you get this ugly message :

nicolau@mars:~$ pip3 list Traceback (most recent call last): File "/home/nicolau/.local/bin/pip3", line 5, in <module> from pip._internal.cli.main import main ModuleNotFoundError: No module named 'pip._internal'

Also on install :

nicolau@mars:~$ pip install Cython Traceback (most recent call last): File "/home/nicolau/.local/bin/pip", line 5, in from pip._internal.cli.main import main ModuleNotFoundError: No module named 'pip._internal' nicolau@mars:~$ sudo pip install Cython

If you want to run the service as root, you have to install the module with sudo: sudo python3 -m pip install my_module

repositories and PPAs

Personal Package Archives (PPAs) are software repositories designed for Ubuntu users and are easier to install than other third-party repositories.

$ /etc/apt/sources.list $ cat /etc/apt/sources.list.d
how to use pip

resum i verify

The correct form is

python3 -m pip install package_name

Si pip ja apunta a python3 llavors podem

pip install requests

Per comprovar on apunta pip :

sebas@minie:~$ pip --version pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10) sebas@minie:~$ pip3 --version pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10) sebas@minie:~$ python3 -m pip --version pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
Under guindows

[20160604] Install under W2008R2 at W500.

Compte : instalar python v2 : 2.7.11 - normal "Windows x86 MSI installer"

Llegir Using Windows

Resultat :

C:\Python27> python --version Python 2.7.11
Using PIP installer under windows

If a name.py code contains a "import" sentence, this "code.py" must be at this directory : "c:\python27\lib"

You can install a module using "c:\python27\scripts\pip install modulename"

C:\Python27\Scripts> .\pip install pyping Collecting pyping Downloading pyping-0.0.5.tar.gz Installing collected packages: pyping Running setup.py install for pyping Successfully installed pyping-0.0.5 You are using pip version 7.1.2, however version 9.0.1 is available. You should consider upgrading via the 'python -m pip install --upgrade pip' command.

Lets update "pip" :

C:\> python -m pip install --upgrade pip Collecting pip Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB) 100% |################################| 1.3MB 292kB/s Installing collected packages: pip Found existing installation: pip 7.1.2 Uninstalling pip-7.1.2: Successfully uninstalled pip-7.1.2 Successfully installed pip-9.0.1

Lets visit "pyping" again :

C:\> python -m pip install --upgrade pyping Requirement already up-to-date: pyping in c:\python27\lib\site-packages

Use pyping sample.

python on Raspberry

Read about installing software in python

First steps

In my Raspberry we have :

$ python --version Python 2.7.13 $ which python /usr/bin/python $ python2 --version Python 2.7.13 $ which python2 /usr/bin/python2 $ python3 --version Python 3.5.3 $ which python3 /usr/bin/python3

How does linux know which one to use ?

sebas@henry:~ $ ls -l $(which python) lrwxrwxrwx 1 root root 9 Jan 24 2017 /usr/bin/python -> python2.7

To change it, use

sebas@henry:~ $ sudo ln -sfv /usr/bin/python3 $(which python)
update pip under linux

python -m pip install --upgrade pip
Installing packages

I've done :

moving to python v3

Version we have :

nicolau@mars:~ $ python --version Python 3.6.9

Problems and solutions :

We do it on :

SSL problem

On almost any install using pip3 or python3 (requests on R3), I get :

sag@odin:~ $ python3 -m pip install --upgrade pip pip is configured with locations that require TLS/SSL, however the ssl module in python is not available.

OpenSSL versions are :

sag@odin:/usr/local/bin $ python Python 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import ssl >>> ssl.OPENSSL_VERSION 'OpenSSL 1.1.0l 10 Sep 2019' >>> sag@odin:/usr/local/bin $ python3 Python 3.7.0 (default, Jul 10 2020, 18:31:19) [GCC 4.9.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import ssl Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.7/ssl.py", line 98, in <module> import _ssl # if we can't import it, let the error propagate ModuleNotFoundError: No module named '_ssl' >>>

After moving back from 3.7 to 3.5 (see above), ssl works fine :

sag@odin:~ $ python3 Python 3.5.3 (default, Sep 27 2018, 17:25:39) [GCC 6.3.0 20170516] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import ssl >>> ssl.OPENSSL_VERSION 'OpenSSL 1.1.0l 10 Sep 2019' >>>
RRDTOOL problem with python3

*** Cant use rrdtool with python3 ***
why ?

pi@R4:~/python/pkw/bin $ cat pkw_cli.py #!/usr/bin/python2 # # amb python3 tenim # pi@R4:~/python/pkw/bin $ ./pkw_cli.py # Traceback (most recent call last): # File "./pkw_cli.py", line 14, in &module> # import pkw_pkg # File "/home/pi/python/pkw/bin/pkw_pkg/__init__.py", line 7, in &module> # from . import pkw_initialize # File "/home/pi/python/pkw/bin/pkw_pkg/pkw_initialize.py", line 6, in &module> # import rrdtool # ModuleNotFoundError: No module named 'rrdtool'

Here we find

pip install rrdtool

Note: unless binary versions are available for your target system, the command above requires rrdtool development files (headers, libraries, dependencies) to be installed.

See python-rrdtool


python details
script header

A shebang line defines where the interpreter is located

SheBang is ignored if we use this form :

$ python ./my_script.py

The general form ...

#!/usr/bin/env python

... executes whatever python is on the users $PATH. This form is resilient to the python interpreter being moved around, which makes it somewhat more portable, but it also means that the user can override the standard python interpreter by putting something ahead of it in $PATH. Depending on your goals, this behavior may or may not be OK.

The specific form ...

#!/usr/bin/python

... deals with the common case that a python interpreter is installed in /usr/bin. If it's installed somewhere else, you lose. But this is a good way to ensure you get exactly the version you want or else nothing at all ("fail-stop" behavior).

stackoverflow

The most specific form ...

#!/usr/bin/python3

... forces to work with version 3, installed on a specific path {mychoice}

You can use

#!/usr/bin/env python3 # -*- coding: UTF-8 -*-

...for portability across different systems in case they have the language interpreter installed in different locations.

decoding UTF

#!/usr/bin/env python # -*- coding: UTF-8 -*-

encoding declarations

display python version runtime

nicolau@mars:~/sebas/python/veure_versions$ cat mostra_versio_python.py #!/usr/bin/env python # -*- coding: UTF-8 -*- import platform # docs import sys print( "1 python version", platform.python_version() ) print( "2 python version", sys.version ) print( "3 python version", sys.version_info ) sys.exit()

Ens dona :

nicolau@mars:~/sebas/python/veure_versions$ ./mostra_versio_python.py 1 python version 3.7.5 2 python version 3.7.5 (default, Dec 9 2021, 17:04:37) [GCC 8.4.0] 3 python version sys.version_info(major=3, minor=7, micro=5, releaselevel='final', serial=0)

Or maybe

if sys.version_info.major == 3: print('*** python3') else: print('*** python2')

More complete :

pi@R4:~/python $ cat 1_veure_versio_python.py #!/usr/bin/env python # -*- coding: UTF-8 -*- # https://stackoverflow.com/questions/9079036/how-do-i-detect-the-python-version-at-runtime import datetime # for timestamp import sys def check_installation(rv): current_version = sys.version_info print( "CV is {%i/%i}." % (current_version[0], current_version[1]) ) if current_version[0] == rv[0] and current_version[1] >= rv[1]: pass else: sys.stderr.write( "[%s] - error: your python interpreter must be %d.%d or greater\n" % (sys.argv[0], rv[0], rv[1]) ) sys.exit(-1) return 0 if __name__ == "__main__": required_version = (2,7) check_installation(required_version)
python parameters / arguments

Complete code with exceptions

Some simple parameters :

#!/usr/bin/env python # -*- coding: UTF-8 -*- import sys numArg = len(sys.argv) print( 'R4 : /home/pi/eines/5_mostrar_parametres.py amb', numArg, 'parametres' ) idx = 0 for arg in sys.argv: print( 'Idx (%i) : (%s).' % ( idx, arg ) ) idx = idx + 1 print( 'Number of arguments (%i).' % numArg ) print( 'Argument list (%s).' % sys.argv ) Num = 0 if ( numArg > 1 ) : szAra = sys.argv[1] if ( szAra == '?' ) : print( 'provide date param in form "yyyy-mm-dd".' ) sys.exit() myFilename = 'temperatura_' + szAra + '.txt' sys.exit()

There are "non-keyword" parameters, and there are "keyword" parameters.

They have to be in that order, or you get a error message like this :

SyntaxError: non-keyword arg after keyword arg
selecting beVerbose or not from input argument

bVerbose = False numArg = len(sys.argv) if ( numArg > 1 ) : szAra = sys.argv[1] if ( szAra == '+' ) : bVerbose = True ; if bVerbose: print('>>> comencem !')
function arguments and default values

python allows function arguments to have default values. If the function is called without the argument, the argument gets its default value.

Default values indicate that the function argument will take that value if no argument value is passed during the function call.

The default value is assigned by using the assignment(=) operator of the form keywordname=value.

def student(firstname, lastname ='Mark', standard ='Fifth'):
some special operations
"/" vs "//"

/ is regular division (returns float) and // is floor division (returns int).
Floor division was introduced in python 3.

rjust()

right-justify and fill with char - w3schools :

string.rjust(length, character)
standard types that are built into the interpreter

docs.python :

Single quote, double quote, triple quote, reverse quote

Generally, double quotes are used for string representation and single quotes are used for regular expressions, dict keys or SQL - geeks 4 geeks

Double quotes around a string are used to specify a regular expression search - ks.uiuc

When a string contains special characters like quotes, we need to escape them :

>>> 'It\'s a good example.' "It's a good example."

You can put a backslash character followed by a quote (\" or \'). This is called an escape sequence and python will remove the backslash, and put just the quote in the string.

We can use triple quotes (i.e., triplet of single quotes or triplet double quotes) to represent the strings containing both single and double quotes to eliminate the need of escaping any :

>>> print('''She said, "Thank you! It's mine."''') She said, "Thank you! It's mine."

A pending improvement : template strings

console.log( '*** (' + genTimeStamp() + ') *** index DOM ready.' ) ; console.log( `*** ( ${ genTimeStamp() } ) *** index DOM ready.` ) ; var szTemp = genTimeStamp() + ' *** la temperatura a casa meva es ' + mijson.temp + ' ºC ***’ ; var szTemp = `${ genTimeStamp() } *** la temperatura a casa meva es ${ mijson.temp } ºC ***` ;

Compte : removed backticks in python 3 !

curios problema de'n Ramon

Les cometes han de ser senzilles - si les posem dobles, tenim :

SERVER_HOST = "fronius" --- value error : host can't be set (not a valid IP address or hostname)

python procedures and parameters

We can define some code to be used few times

#define a procedure def procedureName(arg1, arg2, ...): print('put instructions here') #call the procedure procedureName()

easy python docs

python procedures return values

A python function will always have a return value. That default return value will always be None.

real python


Dealing with environment variables

To display all environment, do :

$ printenv or $ env

To read an environment variable from python, use os.environ :

import os my_host = os.environ["HOSTNAME"] # can produce

library

Compte : "HOSTNAME" is NOT an envir var ! pubs.opengroup

Get it from "socket" :

import socket szHost = socket.gethostname()
How to set a global envir var

Define it in "/etc/environment" and reboot :

sebas@pi0alby:~ $ cat /etc/environment SAG_tg_tkn="1234567890987654321" sebas@pi0alby:~/ $ echo $SAG_tg_tkn 1234567890987654321

Cant "export" it :

sebas@pi0alby:~ $ cat /etc/environment export SAG_tg_tkn="1234567890987654321" /vat/log/syslog : Sep 16 01:11:32 pi0alby systemd[527]: /usr/lib/environment.d/99-environment.conf:2: invalid variable name "export SAG_tg_tkn", ignoring.

Verify it is visible :

pi@pi0:~/python/bot $ python3 Python 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.environ['MYUSR'] 'myusr' >>> os.environ['MYPWD'] 'mypwd' >>> os.environ['MYTOKEN'] '1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghij'
How to read a global envir var into python

Use "os" :

sebas@pi0alby:~/python/envir $ cat 6_llegir_token.py #!/usr/bin/env python import os import sys my_token = os.environ["SAG_tg_tkn"] lng = len( my_token ) print( 'llegit token [', my_token, '].' ) print( 'llargada [', lng, '].' ) sys.exit()

Aware of errors, "None" versus "KeyError" :

my_bot_token = os.environ['SAG_PI0_TOKEN'] # if non-existing, you get 'KeyError' exception my_bot_token = os.environ.get('SAG_PI0_TOKEN') # using get will return 'None' if a key is not present rather than raise a 'KeyError' try: my_bot_token = os.environ["SAG_PI0_TOKEN"] except Exception as e: print("{0}".format(e.__class__))
read a global envir var, with default value if non-existing

DEVICE_ID = "35680858b4e62d74d557" # endoll ID DEVICE_IP = "192.168.1.100" # endoll IP DEVICE_KEY = "uxyew1YTvqqx8Yre" # endoll KEY - see "python3 -m tinytuya wizard" DEVICE_VERS = "3.3" # check for environmental variables and always use those if available PLUGID = os.getenv('PLUGID', DEVICE_ID) PLUGIP = os.getenv('PLUGIP', DEVICE_IP) PLUGKEY = os.getenv('PLUGKEY', DEVICE_KEY) PLUGVERS = os.getenv('PLUGVERS', DEVICE_VERS) # OS = miscellaneous operating system interfaces

Lets improve the code with a "try" sentence :

try: myIP = os.environ["SAG_ENDOLL_SP21_IP"] print( '>>> envir IP is {}'.format(myIP) ) PLUGIP = os.getenv('SAG_ENDOLL_SP21_IP', DEVICEIP) print( '>>> default IP is {}'.format(DEVICEIP) ) print( '>>> IP in use is {}'.format(PLUGIP) ) except KeyError as e: print( '--- got a KeyError - reason {%s}' % str(e) ) print( '--- undefined SAG_ENDOLL_SP21_IP' ) except AttributeError as e: print( "--- Attribute Error (%s) " % ( e ) ) sys.exit()
How to read envir var into python from a service

From a service, /etc/environment is no used.

So, we use EnvironmentFile [***]


How to run python code as a service
How to log from run python code when running as a service

from systemd import journal journal.write("Hello Lennart") ModuleNotFoundError: No module named 'systemd' python3 -m pip install systemd

Display your log lines using journalctl

sebas@r4:~/python/telegram/telepot $ sudo journalctl --lines 0 --follow _SYSTEMD_UNIT=r4_bot.service

sys.exc_info

sys.exc_info docu

Un molt bon article de linuxjournal.com/article/5821 : sys.exc_info() function

try: ... except SomeException: tb = sys.exc_info()[2] raise OtherException(...).with_traceback(tb)

Un altre StackOverflow :

import sys import traceback def formatExceptionInfo(maxTBlevel=5): cla, exc, trbk = sys.exc_info() excName = cla.__name__ try: excArgs = exc.__dict__["args"] except KeyError: excArgs = "<no args>" excTb = traceback.format_tb(trbk, maxTBlevel) return (excName, excArgs, excTb) try: x = x + 1 except: print formatExceptionInfo()

Code from linuxjournal.com/article/5821 !!!


Handling exceptions

Exceptions are used to handle errors and other exceptional conditions that may occur during program execution.

Catching Any Exception:

try: value = 10 / 0 # some code that might raise an exception except Exception as e: print(f"An error occurred: {e}")

See exception hierarchy and KeyError

wiki :

try: print("MY_HOME:", os.environ['MY_HOME']) except KeyError as ke: print( '--- got a KeyError {%s}' % str(ke) ) except AttributeError as ae: print( "--- got a Attribute Error {%s}" % ae ) except NameError as ne: print( "--- Name error: ", ne ) except: print( "--- unexpected error: ", str( sys.exc_info()[0] ) )

A more complete example is :

try: while True: some_code except ( KeyboardInterrupt, SystemExit ) : exit_code_here

Nice code :

import os # import the os module path = '/var/www' try: os.chdir(path) print ( "+++ Current working directory: {0}".format(os.getcwd() ) ) except FileNotFoundError: print("--- Directory: {0} does not exist".format(path)) except NotADirectoryError: print("--- {0} is not a directory".format(path)) except PermissionError: print("--- You do not have permissions to change to {0}".format(path)) except OSError as err : # https://docs.python.org/3/tutorial/errors.html print(err) print("--- OS error: {0}".format(err))

linuxize

A bit more could be :

try: doSomeEvilThing() except Exception, e: handleException(e) raise # "raise" without passing an exception object causes the original traceback to be preserved # typically it is much better than "raise e"

Complete one :

nicolau@mars:~/sebas/ramon/minicom_test$ cat 0_test.py #!/usr/bin/env python import sys try: x = x + 1 except: print( sys.exc_info() ) nicolau@mars:~/sebas/ramon/minicom_test$ ./0_test.py (<class 'NameError'>, NameError("name 'x' is not defined",), <traceback object at 0x7f66b016bb48>)

GPT says "basic syntax of exception handling"

try: # Code that might raise an exception except SomeException as e: # Code that runs if an exception is raised print(f"An error occurred: {e}") else: # Code that runs if no exception occurs print("No errors occurred!") finally: # Code that always runs, regardless of whether an exception occurred or not print("This will always run.")

A larger one :

try: file2 = urllib.request.Request('site goes here') file2.add_header("User-Agent", 'Opera/9.61 (Windows NT 5.1; U; en) Presto/2.1.1') ResponseData = urllib.request.urlopen(file2).read().decode("utf8", 'ignore') except socket.error as e: ResponseData = '' except socket.timeout as e: ResponseData = '' except UnicodeEncodeError as e: ResponseData = '' except http.client.BadStatusLine as e: ResponseData = '' except http.client.IncompleteRead as e: ResponseData = '' except urllib.error.HTTPError as e: ResponseData = '' except urllib.error.URLError as e: ResponseData = e.reason

be careful with catching the subclass of errors before their superclass. In your example, this would mean putting HTTPError before URLError. Otherwise, the subclass will never get caught.

The last except clause may omit the exception name(s), to serve as a wildcard :

try: except: print( "--- Unexpected error: ", str( sys.exc_info()[0] ) ) print( "--- Unexpected error: ", sys.exc_info() ) --- Unexpected error: (<class 'NameError'>, NameError("name 'psutil' is not defined",), <traceback object at 0x7f0f5aaf9b48>)

errors

Another source :

You can either catch the base-class exception, which will handle all cases :

try: r = requests.get(url, params={'s': thing}) except requests.exceptions.RequestException as e: # this is the correct syntax - systemOverflow raise SystemExit(e)

Or you can catch them separately and do different things :

try: r = requests.get(url, params={'s': thing}) except requests.exceptions.Timeout: # maybe set up for a retry, or continue in a retry loop except requests.exceptions.TooManyRedirects: # tell the user their URL was bad and try a different one except requests.exceptions.RequestException as e: raise SystemExit(e) # catastrophic error - bail.

read the docs - requests errors and exceptions

No puc treure els detalls de un "Name Error" : docs.python , airbrake

Tampoc de un "TypeError" - code

except Exception as e: print( 'Exception ' + str(e) ) # "e" te tipus "TypeError"
exemple complert errors requests.get

nicolau@mars:~/sebas/python/llegir_preu_electre/agafen_altres_dades_de_REE$ cat 10073_eolica.py import json try: response = requests.get(url, headers=headers) response.raise_for_status() # we want http errors (e.g. 401 Unauthorized) to raise exceptions if response.status_code == 200: json_data = json.loads(response.text) # parse a JSON string and convert into a dictionary json_string = json.dumps(json_data, indent=4) # convert to a string print ( '+++ JSON [%s].' % json_string ) else: print( "--- status not 200 error: ", response.status_code ) # go from specific to general down the stack of errors except requests.exceptions.HTTPError as errh: # python 3 only print ("--- Http Error:", errh) raise SystemExit(errh) except requests.exceptions.ConnectionError as errc: print ("--- Error Connecting:", errc) except requests.exceptions.Timeout as errt: print ("--- Timeout Error:", errt) except requests.exceptions.TooManyRedirects: # tell the user their URL was bad and try a different one print ( '--- redirect.') except requests.exceptions.RequestException as errex: # python 3 only raise SystemExit(errex)
SIGINT, Keyboard interrupt

import time, sys x = 1 while True: try: print( x ) time.sleep(.3) # wait() a little bit x += 1 except KeyboardInterrupt: print( "--- kbd Ctrl-C : bye" ) sys.exit()

stacloverflow

TypeError exception

try: except TypeError as te: print(f"--- a TYPE error occurred: {te}")
ValueError exception

try: except ValueError as ve: print( "--- Value Error : ", ve )
NameError exception

In my_main() we can get this error message :

--- Unexpected error: <class 'NameError'>

Then, we have to code :

except NameError as ne: print( "--- Name error: ", ne ) # output expected NameErrors

And now we can see the real cause of the error :

--- Name error: name 'psutil' is not defined

Resum : faltava "import psutil"

ALL exceptions

"complete hierarchy" from here from here

BaseException +-- SystemExit +-- KeyboardInterrupt +-- GeneratorExit +-- Exception +-- StopIteration +-- StopAsyncIteration +-- ArithmeticError | +-- FloatingPointError | +-- OverflowError | +-- ZeroDivisionError +-- AssertionError +-- AttributeError +-- BufferError +-- EOFError +-- ImportError +-- ModuleNotFoundError +-- LookupError | +-- IndexError | +-- KeyError +-- MemoryError +-- NameError | +-- UnboundLocalError +-- OSError | +-- BlockingIOError | +-- ChildProcessError | +-- ConnectionError | | +-- BrokenPipeError | | +-- ConnectionAbortedError | | +-- ConnectionRefusedError | | +-- ConnectionResetError | +-- FileExistsError | +-- FileNotFoundError | +-- InterruptedError | +-- IsADirectoryError | +-- NotADirectoryError | +-- PermissionError | +-- ProcessLookupError | +-- TimeoutError +-- ReferenceError +-- RuntimeError | +-- NotImplementedError | +-- RecursionError +-- SyntaxError | +-- IndentationError | +-- TabError +-- SystemError +-- TypeError +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- PendingDeprecationWarning +-- RuntimeWarning +-- SyntaxWarning +-- UserWarning +-- FutureWarning +-- ImportWarning +-- UnicodeWarning +-- BytesWarning +-- ResourceWarning

var scope and global vars

Python has no command for declaring a variable. A variable is created the moment you first assign a value to it.

nicolau@mars:~/sebas/python/proves$ cat global_var.py #!/usr/bin/env python3 import sys Idx = 0 def my_func() : global Idx # (***) Idx = 6 return Idx RC = my_func() print( "{%i-%i}" % (Idx, RC) )

Result is {0-6} or {6-6} depending on the presence of the {***} sentence


logging

sebas@pi0alby:~/python/telegram $ cat client.py #!/usr/bin/env python3 import logging import sys import requests # python -m pip install requests fnLog = '/home/sebas/logs/send_telegram.log' szTxt = " *** R0 envia al bot del TG *** " logging.basicConfig( filename=fnLog, level=logging.DEBUG, \ format='%(asctime)s %(message)s', datefmt='%d/%m/%Y %H:%M:%S' ) logging.info( '\n\n *** Anem a enviar al bot.' ) logging.warning( 'Texte {%s}', szTxt ) tg_rc = telegram_bot_sendtext( szTxt ) logging.debug( f'RC {tg_rc}' ) sys.exit()

logging basics

logging to a file

own log or bitacora

En lloc de fer "print()", cridem al nostre codi per afegir marca temporal :

def trassa( szTxt ): now = datetime.datetime.now() szNow = now.strftime("%Y-%m-%d %H:%M:%S ") print( szNow + szTxt )

atexit()

atexit - this only works with functions that can be called without arguments.

sebas@pi0alby:~/python $ cat mostrar_temperatura.py #!/usr/bin/env python import atexit # exit hook import datetime # timestamp import sys # to "print" def all_done(): print( 'all_done()' ) print '>>> Registering' atexit.register( all_done ) print '+++ Registered' now = datetime.datetime.now() szTime = now.strftime("%Y-%m-%d %H:%M:%S %Z") temp = int(open('/sys/class/thermal/thermal_zone0/temp').read()) / 1e3 print( '[' + szTime + '] temp.CPU => ' + str(temp).' ) print 'Exiting...' sys.exit()

Non-Ascii characters

Poso una "ç" en un comentari i surt :

File "./bot_pihole.py", line 35 SyntaxError: Non-ASCII character '\xc3' in file ./bot_pihole.py on line 35, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

Llegim http://python.org/dev/peps/pep-0263/


"if ... then ... else" statement

The complete form is :

if b > a: print("b is greater than a") < more statements > else: # indentation defines the begin and end of the "then" and "else" clauses print("b is not greater than a") < other statements >

A short form in one line is :

if len(szH) == 1: szH = '0' + szH

I use a lot :

bVerbose = False # True if bVerbose: print('>>> llegir la temperatura')

You can put few statements in one line :

if (j == 8): szOut += '\n' ; j = 0 ;

In python, you can have if-else statements on one line, using the conditional expression syntax:

some_expression if condition else other_expression

codingem

A compound if statement is written like this :

if (a > b) and (c < d):
nested "if" statements

if iVal == 0: print("+++ select an operation") elif iVal == 1: print("+++ display switch status") elif iVal == 2: print("+++ set switch ON") else: print("--- invalid selection {}".format(iVal) )

python while loop

A simple loop is :

i = 1 while i < 6: print(i) i += 1

A "forever" loop is like this :

sebas@pi0alby:~/python/i2c $ cat MCP23008_flash.py #!/usr/bin/python3 # MCP23008 address, 0x20(32) # registre 9 = gpio data register import smbus import time bus = smbus.SMBus(1) # set I2C bus bus.write_byte_data(0x20, 0x00, 0x00) # all pins are configured as outputs while True: bus.write_byte_data(0x20, 0x09, 0x00) time.sleep(0.5) # wait(), delay next operation bus.write_byte_data(0x20, 0x09, 0xFF) time.sleep(0.5) # delay

"print()" statement, python v3

In python 3, print() is a function

Syntax is (few samples) :

print 'tenim ' + str(nummsg) + ' msg' # straight print 'Msg ID is (%s/%s).' % (num, nummsg) # 2 params print( 'The value of PI is approx %5.3f.' % math.pi ) # format print( 'Disk total (%d), free (%d).' % ( my_disk_total, my_disk_free ) ) # 2 parameters szName = "My name is {}, I'm {}".format( "John", 36 ) # el millor metode txt1 = "My name is {fname}, I'm {age}".format(fname = "John", age = 36) # named indexes agent_name = 'James Bond' kill_count = 9 # old ways print("%s has killed %d enemies" % (agent_name,kill_count)) print('{} has killed {} enemies'.format(agent_name,kill_count)) print('{name} has killed {kill} enemies'.format(name=agent_name,kill=kill_count)) # f-strings way print(f'{agent_name} has killed {kill_count} enemies')

Mind that the "print(" initial parentheses has to be closed at the very end of line, after the "% (params)"

format specifiers

Lots of print format tricks :

'{1} {0}'.format('one', 'two') -> 'two one'

The %s specifier converts the object using str(), and %r converts it using repr() - url

how to format integers, strings, etc

hv, lv = tuple( struct.pack('>H', v ) ) # split an integer (2 bytes) into 2 # "<" is Little Endian, ">" is Big Endian chH = f'{ hv:02X }' # convert integer into hex, specifying length

"pass" statement

pass statement


"as" sentence

El as es per fer un alias, i que no hagis de canviar totes les referencies en cas que canviis la llibreria. Molt util per saltar de versions.

import sebastia as sebas sebas.desperta() sebas.festa_major()

Pero de cop ara ets el sebastianot oi?

import sebastianot as sebas

Compte : "old" name is not valid anymore, so we can find this error :

C:\python\> type nmpy.py import numpy as np print "NumPy version %s" % numpy.__version__ C:\python> python nmpy.py Traceback (most recent call last): File "nmpy.py", line 3, in <module> print "NumPy version %s" % numpy.__version__ NameError: name 'numpy' is not defined

python "bytes" object

A "bytes" object represents a sequence of bytes, a "string" object represents a sequence of unicode code points.

To convert between from "bytes" to "string" and from "string" back to "bytes" you use the bytes.encode and string.decode functions. These functions take two parameters, an encoding and an error handling policy.


python circular list

"itertools" or this

conn = ['a', 'b', 'c', 'd', 'e', 'f'] conn_len = len(conn) index = 0 while True: print(conn[index]) index = (index + 1) % conn_len

python collection data types

There are four collection data types in the python programming language:

w3 schools

Use type() function to print the data type of a var :

data = d.status() if (d): print( type( data['dps']['1'] ) ) # output is ... <class 'bool'> bVal = data['dps']['1'] #

In python programming, key-value pairs are dictionary objects and ordered list are list objects.

Brief refresher on the techniques used to extract data from a JSON structure.

Data Type Example Extraction Method Python Code ---------- -------- ------------------ ------------ List animal = [cat, dog. horse. bird] Index operator with square brackets [] print(animal[0]) # output: 'cat' Dictionary student = {'name': Joe, 'age': 26} Keys with square brackets [] print(student['name']) # output: 'Joe'
python lists

A list is a collection which is ordered and changeable. In python lists are written with square brackets.

Each item in the list has a value (color name) and an index (its position in the list). Python uses zero-based indexing.

thislist = [ "apple", "banana", "cherry" ] print(thislist)
length of a list

inp_lst = [ 'Python', 'Java', 'Kotlin', 'Machine Learning', 'Keras' ] size = len(inp_lst) print(size)
looping through a list

Print all items in the list :

thislist = [ "apple", "banana", "cherry" ] for x in thislist: print(x)

Print all items by referring to their index number:

thislist = [ "apple", "banana", "cherry" ] for i in range( len( thislist )): print( thislist[i] )

Print all items, using a while loop to go through all the index numbers

thislist = [ "apple", "banana", "cherry" ] i = 0 while i < len( thislist ): print( thislist[i] ) i = i + 1
python tuples

A tuple is a collection which is ordered and unchangeable. In python tuples are written with round brackets.

thistuple = ( "apple", "banana", "cherry" ) print(thistuple)

A real sample :

sebas@pi0alby:~/python/tinet $ ./get_internal_ip.py IPs of network adapter wlan0 ip type [<class 'str'>] ip0 [192.168.1.222] pfx 24
python sets

A set is a collection which is unordered and unindexed. In python sets are written with curly brackets.

thisset = { "apple", "banana", "cherry" } print(thisset)

The reason the set iteration order changes from run-to-run appears to be because Python uses hash seed randomization by default. stackOverflow

python set length

# creating a set myset = {"USA", "CANADA", "BRAZIL"} # implementing the len() function print(len(myset))

educative.io

iterate over a python set

fruits = { "Apple", "Peach", "Mango" } for fruit in fruits: print(fruit)

programiz

python dictionaries

A dictionary is a collection which is unordered, changeable and indexed. In python dictionaries are written with curly brackets, and they have keys and values.

thisdict = { "brand": "Ford", "model": "Mustang", "year": 1964 } print(thisdict) objChat = { 'first_name': 'Sebastia', 'id': 304588090, 'last_name': 'Altemir', 'type': 'private' } print ("Original dictionary is : " + str(objChat)) # print the dictionary
an overview of keys and values in dictionaries in python

Keys inside a python dictionary can only be of a type that is immutable.

Immutable data types in python are integers, strings, tuples, floating point numbers, and Booleans.

Dictionary keys cannot be of a type that is mutable, such as sets, lists, or dictionaries.

When it comes to values inside a python dictionary there are no restrictions.

Another thing to note about the differences between keys and values in python dictionaries, is the fact that keys are unique. This means that a key can only appear once in the dictionary, whereas there can be duplicate values.

freecodecamp

create an empty dictionary and add elements to it

my_dict = {} # create an empty dictionary ... my_dict = dict() # another way : use the dict() function without passing any arguments my_dict[my_key] = my_val # afegim al nou diccionari
change values in a dictionary

You can change the value of a specific item by referring to its key name:

thisdict = { "brand": "Ford", "model": "Mustang", "year": 1964 } thisdict["year"] = 2018
looping through a dictionary

You can loop through a dictionary by using a for loop.
When looping through a dictionary, the return value are the keys of the dictionary, but there are methods to return the values as well.

Some details : url

objChat = { 'first_name': 'Sebastia', 'id': 304588090, 'last_name': 'Altemir', 'type': 'private' } print ("Dict key-value are : ") for i in objChat : # using in operator to get key and value print(i, objChat[i]) Dict key-value are : id 304588090 first_name Sebastia last_name Altemir type private
accessing a specific dictionary element

objChat = { 'first_name': 'Sebastia', 'id': 304588090, 'last_name': 'Altemir', 'type': 'private' } myId = objChat['id'] myUserName = objChat['last_name'] myFirstName = objChat['first_name']
delete a dictionary entry : "del()" or "pop()" ?

geeks

delete a specific dictionary element

Have to create a "shadow" copy not to get "RuntimeError: dictionary changed size during iteration" :

my_dict = { 'a': 1, 'b': 2, 'c': 3 } for key in my_dict.copy(): print(key) if key == 'b': del my_dict[key] print(my_dict) # 👉️ {'a': 1, 'c': 3}

bobby hadz

sort

Sorting a python dictionary: values, keys and more


file modes in file handling

r Open text file for reading r+ Open for reading and writing w Create text file for writing w+ Open for reading and writing a Open for writing/append The file is created if it does not exist The stream is positioned at the end of the file a+ Open for reading and writing The file is created if it does not exist The stream is positioned at the end of the file +++ r open for reading. The file pointer is placed at the beginning of the file. This is the default mode. r+ Opens a file for both reading and writing. The file pointer will be at the beginning of the file. w Opens a file for writing only. Overwrites the file if the file exists. If the file does not exist, creates a new file for writing. w+ Opens a file for both writing and reading. Overwrites the existing file if the file exists. If the file does not exist, it creates a new file for reading and writing. rb Opens a file for reading only in binary format. The file pointer is placed at the beginning of the file. rb+ Opens a file for both reading and writing in binary format. wb+ Opens a file for both writing and reading in binary format. Overwrites the existing file if the file exists. If the file does not exist, it creates a new file for reading and writing. a Opens a file for appending. The file pointer is at the end of the file if the file exists. If the file does not exist, it creates a new file for writing. ab Opens a file for appending in binary format. The file pointer is at the end of the file if the file exists. If the file does not exist, it creates a new file for writing. a+ Opens a file for both appending and reading. The file pointer is at the end of the file if the file exists. If the file does not exist, it creates a new file for reading and writing. ab+ Opens a file for both appending and reading in binary format. The file pointer is at the end of the file if the file exists. If the file does not exist, it creates a new file for reading and writing. x open for exclusive creation, failing if the file already exists (Python 3)
Read / write files
write to a file

Simple way :

with open( szFQFN, "a+" ) as fOut: fOut.write( szOut ) # .close() called automatically

Complete :

fd = open( "demofile2.txt", "a" ) # append mode fd.write("Now the file has more content!") fd.close()
read from a file

fd = open('myfile.txt', 'r')

Elegant :

try: reader = open('dog_breeds.txt') # further file processing goes here finally: reader.close()

Another :

try: f = open(arg, 'r') except OSError: print('--- cannot open', arg) else: print(arg, 'has', len(f.readlines()), 'lines') f.close()

Working with : CSV, JSON

llegim un fitxer linia a linia

#!/usr/bin/env python # -*- coding: UTF-8 -*- iFN = 'in.txt' fd = open( iFN, 'r' ) count = 0 while True: line = fd.readline() # get next line from file if not line: # if line is empty break # .. then end of file is reached -> break the loop # .. else count += 1 print( "Line {}: [{}].".format( count, line.strip() ) ) fd.close()
convertir CSV a diccionari

MY_SAMPLE.CSV a,1 b,2 d = dict() f = open("my_sample.csv") for line in f: line = line.strip() # remove trailing \n (key, val) = line.split(",") # split the line d[key] = val print(d) OUTPUT {'a': '1', 'b': '2'}

adam smith.haus


how to parse a string

parse a string

szNumMsg = str(nummsg) # string is ['80'] newMsgNum = szNumMsg.split("'")[1] # got plain "80" (key, PP, PC, PD) = line.split(",") # split CSV into 3 vars

See split CSV

use "partition"

If it's always going to be an even LHS/RHS split, you can also use the partition method that's built into strings.
It returns a 3-tuple as (LHS, separator, RHS) if the separator is found, and (original_string, '', '') if the separator wasn't present:

>>> "2.7.0_bf4fda703454".partition('_') ('2.7.0', '_', 'bf4fda703454')

and/or

>>> "shazam".partition("_") ('shazam', '', '')

STACKOVERFLOW

Another sample :

el@apollo:~/foo$ python >>> mystring = "theres coffee in that nebula" >>> mytuple = mystring.partition(" ") >>> print type(mytuple) <type 'tuple'> >>> print mytuple ('theres', ' ', 'coffee in that nebula') >>> print mytuple[0] theres >>> print mytuple[2] coffee in that nebula

convert TXT/LOG to HTML

header = '<!doctyle html><html><head><title>My Title</title></head><body>' body = '<table><thead><tr><th>Col 1</th><th>Col 2</th></tr>' footer = '</table></body></html>' with open('input.txt', 'r') as input, open('output.html', 'w') as output: output.writeln(header) output.writeln(body) for line in input: col1, col2 = line.rstrip().split() output.write('<tr><td>{}</td><td>{}</td></tr>\n'.format(col1, col2)) output.write(footer)

stackOverflow


PEP8

PEP8 is the officially suggested convention for code style.

The most widely used style guide in Python is known as PEP8, and can be found at https://www.python.org/dev/peps/pep-0008/

Nice article


python debug

Use python -m pdb <program name> to go step-by-step thru your program.

Complete documentation

Basic debug commands :

C:\sebas\miscosas\albert\python\webcam> python -m pdb 3-v-en-color.py > c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(1)() -> import numpy as np (Pdb) next > c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(2)() -> import cv2 (Pdb) next > c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(6)() -> cap = cv2.VideoCapture(0) (Pdb) next > c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(8)() -> while(True): (Pdb) next > c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(11)() -> ret, frame = cap.read() (Pdb) next > c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(17)() -> if ret: (Pdb) next > c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(18)() -> print dir ( frame ) ; (Pdb) next ['T', '__abs__', '__add__', '__and__', '__array__', '__array_finalize__', '__array_interface__', '__array_prepare__', '_ _array_priority__', '__array_struct__', '__array_wrap__', '__class__', '__contains__', '__copy__', '__deepcopy__', '__de lattr__', '__delitem__', '__delslice__', '__div__', '__divmod__', '__doc__', '__eq__', '__float__', '__floordiv__', '__f ormat__', '__ge__', '__getattribute__', '__getitem__', '__getslice__', '__gt__', '__hash__', '__hex__', '__iadd__', '__i and__', '__idiv__', '__ifloordiv__', '__ilshift__', '__imod__', '__imul__', '__index__', '__init__', '__int__', '__inver t__', '__ior__', '__ipow__', '__irshift__', '__isub__', '__iter__', '__itruediv__', '__ixor__', '__le__', '__len__', '__ long__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__', '__neg__', '__new__', '__nonzero__', '__oct__', '__or__ ', '__pos__', '__pow__', '__radd__', '__rand__', '__rdiv__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', ' __rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', ' __rtruediv__', '__rxor__', '__setattr__', '__setitem__', '__setslice__', '__setstate__', '__sizeof__', '__str__', '__sub __', '__subclasshook__', '__truediv__', '__xor__', 'all', 'any', 'argmax', 'argmin', 'argpartition', 'argsort', 'astype' , 'base', 'byteswap', 'choose', 'clip', 'compress', 'conj', 'conjugate', 'copy', 'ctypes', 'cumprod', 'cumsum', 'data', 'diagonal', 'dot', 'dtype', 'dump', 'dumps', 'fill', 'flags', 'flat', 'flatten', 'getfield', 'imag', 'item', 'itemset', 'itemsize', 'max', 'mean', 'min', 'nbytes', 'ndim', 'newbyteorder', 'nonzero', 'partition', 'prod', 'ptp', 'put', 'ravel ', 'real', 'repeat', 'reshape', 'resize', 'round', 'searchsorted', 'setfield', 'setflags', 'shape', 'size', 'sort', 'squ eeze', 'std', 'strides', 'sum', 'swapaxes', 'take', 'tobytes', 'tofile', 'tolist', 'tostring', 'trace', 'transpose', 'va r', 'view'] > c:\sebas\miscosas\albert\python\webcam\3-v-en-color.py(19)() -> cv2.imshow('3.py - frame',frame) (Pdb)
python breakpoint()

Insert "breakpoint()" to enter debug - realpython

python trace statement execution

The trace module can be invoked from the command line. It can be as simple as python -m trace --count -C . somefile.py ...

doc.python.org {amb programmatic interface}, pymotw.com

python debug links

Here's a list of resources to get started with the python debugger:

print dir

Display built-in functions, methods, modules and module attributes.

$ python -v >>> import ModbusClient Traceback (most recent call last): ModuleNotFoundError: No module named 'ModbusClient' >>> from pyModbusTCP.client import ModbusClient >>> print(dir(ModbusClient))
encoding

Problema 20210227 :

sebas@pi0alby:~/python/tinet $ python -m pdb ./enviar_html_a_tinet.py Traceback (most recent call last): File "/usr/lib/python2.7/pdb.py", line 1314, in main pdb._runscript(mainpyfile) File "/usr/lib/python2.7/pdb.py", line 1233, in _runscript self.run(statement) File "/usr/lib/python2.7/bdb.py", line 400, in run exec cmd in globals, locals File "<string>", line 1, in <module> File "./enviar_html_a_tinet.py", line 57 SyntaxError: Non-ASCII character '\xc3' in file ./enviar_html_a_tinet.py on line 57, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

Solucio - posar com a segona linia el seguent :

# -*- coding: UTF-8 -*-

user-defined functions

A function is a block of organized, reusable code that is used to perform a single, related action. Functions provide better modularity for your application and a high degree of code reusing.

All parameters (arguments) in the python language are passed by reference.

tutorials point , code mentor

See r/w JSON sample

default parameters

Often when defining a function, there are certain values that we want the function to use most of the time, but we'd also like to give the user some flexibility. In this case, we can use default values for arguments.

The values can also be specified by name if desired, in which case the order of the named values does not matter.

lambda functions

Another way of defining short, one-off functions

add = lambda x, y: x + y add(1, 2)

This lambda function is roughly equivalent to

def add(x, y): return x + y

So why would you ever want to use such a thing? Primarily, it comes down to the fact that everything is an object in Python, even functions themselves! That means that functions can be passed as arguments to functions.

WhirlWindTour ch.08


Calling system functions

We shall use subprocess :

#!/usr/bin/env python import subprocess import os import sys # res = subprocess.Popen(['ls','-al','/ahome'],stdout=subprocess.PIPE,stderr=subprocess.PIPE); # output,error = res.communicate() # if res.returncode: # #raise Exception(error) # print "error>>>> ",res.returncode # else: # print "output>>>> ",output try: res = subprocess.Popen(['ls','-al','/home'],stdout=subprocess.PIPE,stderr=subprocess.PIPE); output,error = res.communicate() if output: print "ret> ", res.returncode print "OK> output ", output if error: print "ret> ", res.returncode print "Error> error ", error.strip() #except CalledProcessError as e: # print "CalledError > ",e.returncode # print "CalledError > ",e.output except OSError as e: print "OSError > ", e.errno print "OSError > ", e.strerror print "OSError > ", e.filename except: print( "Error > " + str( sys.exc_info()[0] ) )

gist

Another sample :

mate@punt-omnia:~/eines/python/server_status$ cat sesta.py #!/usr/bin/python # https://docs.python.org/2/contents.html # https://docs.python.org/2/library/subprocess.html import subprocess fname="/home/mate/eines/python/server_status/testfile.txt" mfile = open( fname,"w" ) mfile.write( "+++ (1) Date is\n" ) mcmd = 'date' print 'My comand #1 is [%s].' % mcmd out_date = subprocess.check_output( mcmd ) mfile.write( out_date ) mfile.write( "+++ (2) Temp is\n" ) mcmd = '/usr/bin/sensors' print 'My comand #2 is [%s].' % mcmd out_sensors = subprocess.check_output( mcmd ) mfile.write( out_sensors ) mfile.write( "+++ (3) Fin\n" ) mfile.close()
calling a script with arguments

import subprocess val = subprocess.check_call("./script.sh '%s'" % arg, shell=True)

Remember to convert the args to string before passing, using str(arg).

This can be used to pass as many arguments as desired:

subprocess.check_call("./script.ksh %s %s %s" % (arg1, str(arg2), arg3), shell=True)

I recommend using list as arguments like

subprocess.check_call(["./script.ksh", arg1, arg2, arg3], shell=True)

StackOverflow

my code : calling a script with arguments

import subprocess path_to_local_dir = '/home/pi/python/pkw/eines/analitzar_un_dia' nomShell = 'extreure_1_dia.sh' fnShell = path_to_local_dir + '/' + nomShell szCMD = fnShell + " -d " + szFN + " -i " + "%s" % eINI + " -f " + "%s" % eFIN + " -o " + "%s" % fn_dades_1_dia print ( "CMD ***{}".format (szCMD) ) iRC = subprocess.call( [szCMD], shell=True ) print("Extreure return code", iRC)
System monitoring

Shell scripts for system monitoring from here

cmd = "hostname -I | cut -d' ' -f1" IP = subprocess.check_output(cmd, shell=True).decode("utf-8") cmd = "top -bn1 | grep load | awk '{printf \"CPU Load: %.2f\", $(NF-2)}'" CPU = subprocess.check_output(cmd, shell=True).decode("utf-8") cmd = "free -m | awk 'NR==2{printf \"Mem: %s/%s MB %.2f%%\", $3,$2,$3*100/$2 }'" MemUsage = subprocess.check_output(cmd, shell=True).decode("utf-8") cmd = 'df -h | awk \'$NF=="/"{printf "Disk: %d/%d GB %s", $3,$2,$5}\'' Disk = subprocess.check_output(cmd, shell=True).decode("utf-8")

datetime details

Read datetime docu

strftime
The strftime() method returns a string representing date and time using date, time or datetime object.

now = datetime.datetime.now() # getdatetime object szInici = now.strftime("%Y-%m-%d %H:%M:%S %Z") # convert to string

strptime
The strptime() method creates a datetime object from the given string.

do_Time = datetime.strptime( szTime, "%Y-%m-%dT%H:%M:%S.%f") # datetime object

We can split it into separated values :

import datetime now = datetime.datetime.now() print(now.year, now.month, now.day, now.hour, now.minute, now.second)

And improve it a bit :

year = '{:02d}'.format(now.year) month = '{:02d}'.format(now.month) day = '{:02d}'.format(now.day) hour = '{:02d}'.format(now.hour) minute = '{:02d}'.format(now.minute)

Generate a timestamp

In python, dates are objects. Therefore, when you manipulate them, you manipulate objects, not strings, not timestamps nor anything.

Any object in python have TWO string representations:

ya@punt-omnia:~/eines/monit_campanar$ cat monit_campanar.py import datetime now = datetime.datetime.now() szInici = now.strftime("%Y-%m-%d %H:%M:%S %Z")

saltycrane

Write daily measures into same file and change automagically

pi@R4:~/python/pkw/eines/veure_potencia_instant $ cat 4_veure_potencia_instant_i_esciure_a_fitxer.py szPath_Soroll = '/home/pi/python/soroll/csv' now = datetime.datetime.now() szTemps = now.strftime("%Y-%m-%d %H:%M:%S") # used in filename szHora = now.strftime("%H:%M") # used in CSV szFN_soroll_CSV = "soroll_CSV_%s.csv" % ( now.strftime("%Y_%m_%d") ) # dont use HH ni MM ni SS in Filename - the file contains a full DAY szFQFN_soroll_CSV = szPath_Soroll + '/' + szFN_soroll_CSV szOutCSV = szHora + ',' + str( int(Soroll) ) + '\n' # create a CSV line if beVerbose: print( ">>> (%s) Open (%s)." % (szTemps, szFQFN_soroll_CSV) ) fSorollCSV = open( szFQFN_soroll_CSV, "a+") # open to Append try: fSorollCSV.write( szOutCSV ) # write CSV if beVerbose: print( "+++ (%s) Write (%s)." % (szTemps, szOutCSV) ) finally: fSorollCSV.close() # close file if beVerbose: print( "+++ (%s) Close." % (szTemps) )
Mesurem gap

inici = datetime.datetime.now() szDades = urllib2.urlopen( my_url ).read().decode(encoding='UTF-8').strip() # agafem dades del Fronius final = datetime.datetime.now() delta = final - inici # get difference print( "Time difference is {%s} seconds" % delta.total_seconds() ) # time difference in seconds ms = delta.total_seconds() * 1000 print( "Time difference is {%s} milliseconds" % ms ) # time difference in milliseconds

pynative


Calling python code from bash

You execute a python code from bash like this

python python_script.py arg1 arg2

And to access these arguments in python you can use:

import sys print sys.argv[0] # prints python_script.py print sys.argv[1] # prints arg1 print sys.argv[2] # prints arg2

The python results have to be available to bash :

result=$(python3 python_code.py)

Calling python code from node

Lots of ways and details, and python-shell module and API

Working sample - the python code ( after "npm install python-shell" )

app.get( '/fer_foto', function (req, res) { // var PythonShell = require( 'python-shell' ) ; // send commands to python let {PythonShell} = require( 'python-shell' ) ; // send commands to python since v 1.0.7 var python_options = { // full documentation and source code, github mode: 'text', pythonPath: '/usr/bin/python', pythonOptions: ['-u'], scriptPath: '/home/pi/semafor', args: ['value1', 'value2', 'value3'] } ; console.log( '>>> Hacer foto.' ); PythonShell.run( 'fer_foto.py', python_options, function( err, results ) { if ( err ) { var szErr = '--- Python error. ' ; szErr += 'Path (' + err.path + '). ' ; szErr += 'Stack (' + err.stack + '). ' ; console.log( szErr ) ; throw err ; // fatal error : stop } else { console.log( '(+) Python results are (%j).', results ) ; // results is an array consisting of messages collected during execution png_File = String( results[0] ) ; // convert to string mConsole( "(+) python filename (" + png_File + ")." ) ; res.writeHead( 200, { 'Content-Type': 'application/json' }) ; let myjson = { status: 'OK', imgURL: png_File }; res.end( JSON.stringify( myjson ) ) ; } ; // no error } ) ; } ) ; // fer foto

And the result is :

>>> Hacer foto. (+) Python results are (["hi","kukurutxu"]).
Taking pictures by python using a USB webcam

On ho faig servir (gracies, Albert) :


Calling python code from python

The server.py shall be like this

$ cat server.py def server_job(arg1, arg2, etc): # do whatever the script does retirun iRC if __name__ == "__main__": RC = server_job(sys.argv[1], sys.argv[2], sys.argv[3])

And we can call it like this :

$ cat client.py import server RC = server.server_job(params)

numpy = numerical python

To know the version we have :

import numpy print "### NUMPY version I have is [%s]." % numpy.__version__

Lots of uses

pandas

pandas is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data both easy and intuitive.

pandas doc, options

10 minutes intro

pandas options

pd.set_option('display.max_rows', None) # https://dev.to/chanduthedev/how-to-display-all-rows-from-data-frame-using-pandas-dha pd.set_option('display.max_colwidth', None) # https://www.statology.org/pandas-column-width/

TCP/IP

Consider Results, an elegant and simple HTTP library for python, built for human beings.

TCP/IP :

TCP

pibcast-1.0.tar.gz

get MAC

pi@R4:~$ python3 -m pip install getmac Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple Collecting getmac Downloading https://files.pythonhosted.org/packages/ea/ad/7821cbe5394fcb/getmac-0.8.3-py2.py3-none-any.whl Installing collected packages: getmac The script getmac is installed in '/home/pi/.local/bin' which is not on PATH. Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. Successfully installed getmac-0.8.3

geeks -> es un script !

pi@R4:/usr/local/bin $ cat getmac #!/usr/bin/python3 # -*- coding: utf-8 -*- import re import sys from getmac.__main__ import main if __name__ == '__main__': sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0]) sys.exit(main())
get host name, internal IP, external IP, ...

To get TCP/IP local info into python we use :

# https://pythonhosted.org/ifaddr/ import ifaddr # pip install ifaddr import socket szHost = socket.gethostname() # get "hostname" adapters = ifaddr.get_adapters() for adapter in adapters: print( "IPs of network adapter " + adapter.nice_name ) for ip in adapter.ips: print( " %s/%s" % (ip.ip, ip.network_prefix) ) my_URL = 'http://ipv4.icanhazip.com' my_IP_ext = urlopen( my_URL ).read().decode(encoding='UTF-8').strip() # convert bytes to string and remove trailing \n print( '+++ my ext IP [', my_IP_ext, ']' )
simple IP

There is a simple way to get the IP :

from subprocess import check_output ip4 = check_output(['hostname', '-I']).decode(encoding='UTF-8').strip() # convert bytes to string and remove trailing \n print( ">>> ip4 [%s]." % ip4 )

Explicacion en "man hostname" :

-I, --all-ip-addresses
Display all network addresses of the host. This option enumerates all configured addresses on all network interfaces. The loopback interface and IPv6 link-local addresses are omitted. Contrary to option -i, this option does not depend on name resolution. Do not make any assumptions about the order of the output.

complicated IP

sebas@pi0alby:~/python/i2c/oled $ cat /home/sebas/python/i2c/oled/b_continous_2ip.py # This function allows us to grab any of our IP addresses def get_ip_address( ifname ): s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) return socket.inet_ntoa( fcntl.ioctl( s.fileno(), 0x8915, # SIOCGIFADDR struct.pack("256s", str.encode(ifname[:15])), )[20:24] ) ip_wifi = get_ip_address("wlan0") ip_lan = get_ip_address("eth0")
get wifi name

nicolau@mars:~/sebas/python/ping_GAS2$ cat wifi_name.py #!/usr/bin/env python3 import subprocess import sys subprocess_result = subprocess.Popen('iwgetid',shell=True,stdout=subprocess.PIPE) subprocess_output = subprocess_result.communicate()[0],subprocess_result.returncode network_name = subprocess_output[0].decode('utf-8').strip() print( "name (%s)" % network_name ) sys.exit()
how to ping() from python

nicolau@mars:~/sebas/python/icmp$ cat 1_ping.py #!/usr/bin/env python3 # -*- coding: UTF-8 -*- from icmplib import ping # https://github.com/ValentinBELYN/icmplib myIP = '192.168.1.1' host = ping( myIP, count=4, interval=1, timeout=2, privileged=True) print( ">>> ping IP (%s)." % myIP ) if host.is_alive: print(f'+++ {host.address} is alive! avg_rtt={host.avg_rtt} ms') else: print(f'--- {host.address} is dead')

El resultat es :

nicolau@mars:~/sebas/python/icmp$ sudo ./1_ping.py >>> ping IP (192.168.1.1). +++ 192.168.1.1 is alive! avg_rtt=3.952 ms
icmplib

icmplib at github

pyping

Use it like this :

mate@punt-omnia:~/nodejs-projects/timer$ cat 2_do_ping.py #!/usr/bin/python # we have to return 2 strings : # 1) the IP we are ping-ing # 2) the result import sys import pyping # pip install pyping # https://github.com/certator/pyping/blob/master/bin/pyping numArg = len( sys.argv ) idx = 0 szDesti = sys.argv[1] # in nodejs we code "python_options.args[0] = szIP ;" print 'IP (',szDesti,').' # ret string #1 try: response = pyping.ping( szDesti, timeout=800, count=3 ) # -t 900 -c 2 if response.ret_code == 0: print( "+++ RC 0 - Viagra, reachable" ) # 74.125.143.104 else: print( "--- RC KO - Promescent, unreachable" ) # 1.2.3.4 except ZeroDivisionError: print( "=== RC KO - divBy0" )
how to ftp() from python

STORLINES vs STORBINARY


opening URLs

Lets compare "requests" to "urlopen"

my_url = 'http://ifconfig.me/ip' response = requests.get(my_url) print( '0 my IP : {}'.format( response.text.strip() ) ) my_IP = urlopen( 'http://icanhazip.com' ).read() # get a "bytes" object print( '1 my IP :', my_IP ) my_IP = urlopen( 'http://icanhazip.com' ).read().decode(encoding='UTF-8') # ... convert "bytes" object to string print( '2 my IP :', my_IP ) my_IP = urlopen( 'http://icanhazip.com' ).read().decode(encoding='UTF-8').strip() # ... and remove trailing \n print( '3 my IP :', my_IP ) my_IP = requests.get( "https://api.ipify.org/?format=json" ).json()['ip'] # site provides JSON format print( '4 my IP :', my_IP )

A complete JSON info, as geolocation
homepage , including how to get external IP using 21 languages, as Lua, nim, jQuery, Scala ...

requests

requests is requests : HTTP for Humans - see docu advanced section

Minimum code :

import requests # python -m pip install requests sitestring_input = input("Enter a domain (ie, www.google.com): ") my_url = "http://%s" % [sitestring_input] # getting the URL string formatted safely try: req = requests.get(my_url) except: print("nope, sorry")

Have to add some code to manage errors and exceptions or timeouts

urllib

See code at TG.HTM

import urllib.request import urllib.error import json while True: my_url = input('Enter a url like:(www.google.com)-> ') try: resp = urllib.request.urlopen( my_url, timeout = 5000 ) json_data = json.loads( resp ) except urllib.error.HTTPError as err_h: print(err_h.reason) break # break the loop - otherwise "sys.exit()" except urllib.error.URLError as err_u: print(err_u.reason) break else: print('URL Exists') break

reddit

compte urllib.error sota python 2 o python 3

pi@R4:~$ python Python 2.7.16 (default, Oct 10 2019, 22:02:15) [GCC 8.3.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import urllib.error Traceback (most recent call last): File "&stdin>", line 1, in &module> ImportError: No module named error >>> pi@R4:~/python/pkw/bin/pkw_pkg $ python3 Python 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import urllib.error funciona OK sota python 3 >>>
display module contents - urllib

>>> print(dir(urllib)) ['ContentTooShortError', 'FancyURLopener', 'MAXFTPCACHE', 'URLopener', '__all__', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__version__', '_asciire', '_ftperrors', '_have_ssl', '_hexdig', '_hextochr', '_hostprog', '_is_unicode', '_localhost', '_noheaders', '_nportprog', '_passwdprog', '_portprog', '_queryprog', '_safe_map', '_safe_quoters', '_tagprog', '_thishost', '_typeprog', '_urlopener', '_userprog', '_valueprog', 'addbase', 'addclosehook', 'addinfo', 'addinfourl', 'always_safe', 'base64', 'basejoin', 'c', 'ftpcache', 'ftperrors', 'ftpwrapper', 'getproxies', 'getproxies_environment', 'i', 'localhost', 'noheaders', 'os', 'pathname2url', 'proxy_bypass', 'proxy_bypass_environment', 'quote', 'quote_plus', 're', 'reporthook', 'socket', 'splitattr', 'splithost', 'splitnport', 'splitpasswd', 'splitport', 'splitquery', 'splittag', 'splittype', 'splituser', 'splitvalue', 'ssl', 'string', 'sys', 'test1', 'thishost', 'time', 'toBytes', 'unquote', 'unquote_plus', 'unwrap', 'url2pathname', 'urlcleanup', 'urlencode', 'urlopen', 'urlretrieve']
urllib at Raspberry versus Ubuntu

The same code has to be changed a bit, and I still dont know why : {p2 versus p3 ?}

urllib at Raspberry

pi@R4:~/python/fronius $ cat llegir_energia.py #!/usr/bin/env python # -*- coding: UTF-8 -*- # agafem dades del Fronius import json import urllib hostname = "fronius" # defined in /etc/hosts def main(argv): endpoint = "/solar_api/v1/GetPowerFlowRealtimeData.fcgi" my_url = "http://" + hostname + endpoint szDades = urllib.urlopen( my_url ).read().decode(encoding='UTF-8').strip() objjson = json.loads( szDades ) # parse a JSON string and convert into a dictionary energia_del_dia = objjson["Body"]["Data"]["Inverters"]["1"]["E_Day"] now = datetime.datetime.now() szTime = now.strftime("%Y-%m-%d %H:%M:%S %Z") print( "+++ {%s} todays energy = (%d) {Wh}." % (szTime, energia_del_dia ) ) if __name__ == "__main__": main(sys.argv)
urllib at Ubuntu

nicolau@mars:~/sebas/python/fronius$ cat 8_llegir_energia.py #!/usr/bin/env python # -*- coding: UTF-8 -*- import json # import urllib # raspberry import urllib.request # --- 1st change hostname = "fronius" # defined in /etc/hosts def main(argv): endpoint = "/solar_api/v1/GetPowerFlowRealtimeData.fcgi" my_url = "http://" + hostname + endpoint # szDades = urllib.urlopen( my_url ).read().decode(encoding='UTF-8').strip() # raspberry szDades = urllib.request.urlopen( my_url ).read().decode(encoding='UTF-8').strip() # --- 2nd change objjson = json.loads( szDades ) # parse a JSON string and convert into a dictionary energia_del_dia = objjson["Body"]["Data"]["Inverters"]["1"]["E_Day"] now = datetime.datetime.now() szTime = now.strftime("%Y-%m-%d %H:%M:%S %Z") print( "+++ {%s} todays energy = (%d) {Wh}." % (szTime, energia_del_dia ) ) if __name__ == "__main__": main(sys.argv)
urllib2

urllib2 — extensible library for opening URLs - better known exceptions !

urllib2 provides some extra functionality, namely the urlopen() function can allow you to specify headers

docs.python.org : v 2, v 3

import urllib2 # raspbian - https://docs.python.org/2/library/urllib2.html import json endpoint = "/solar_api/v1/GetMeterRealtimeData.cgi?Scope=System" my_url = "http://" + hostname + endpoint print( "\n>>> (2) Read (%s)." % endpoint ) try: szDades = urllib2.urlopen( my_url, timeout = 5 ).read().decode(encoding='UTF-8').strip() objjson = json.loads( szDades ) # parse a JSON string and convert into a dictionary p_power_real = objjson["Body"]["Data"]["0"]["PowerReal_P_Sum"] print( "+++ real power [GetMeterRealtimeData PowerReal_P_Sum] - (%d) {W}. As P_Grid." % p_power_real ) except urllib2.HTTPError as e: print('HTTP error = ' + str(e.code) ) except urllib2.URLError as e: print('URL error = ' + str(e.reason) ) except KeyboardInterrupt: print( "INTERRUPTED!" )

no funciona amb python3

The urllib2 module has been split across several modules in python 3 named urllib.request and urllib.error - python 3.8 , urllib

my_url = urllib2.Request('http://www.example.com', postBackData, { 'User-Agent' : 'My User Agent' }) try: response = urllib2.urlopen( my_url, timeout = 5 ) except urllib2.HTTPError as e: checksLogger.error('HTTPError = ' + str(e.code)) except urllib2.URLError as e: checksLogger.error('URLError = ' + str(e.reason)) except httplib.HTTPException as e: checksLogger.error('HTTPException') except Exception: import traceback checksLogger.error('generic exception: ' + traceback.format_exc())

stack overrun

urllib2 is the name of the library included in Python 2. You can use the urllib.request library included with Python 3, instead. url

Alternatively, install the requests library for a higher-level and easier to use API. It'll work on both Python 2 and 3

python requests

Requests: HTTP for Humans

Homepage, API docu

requests will work on both Python 2 and 3

Quickstart, advanced documentation

resp = requests.get('https://api.github.com/events') resp = requests.post('https://httpbin.org/post', data={'key': 'value'}) resp = requests.put('https://httpbin.org/put', data={'key': 'value'}) resp = requests.delete('https://httpbin.org/delete') resp = requests.head('https://httpbin.org/get') resp = requests.options('https://httpbin.org/get')

Here is a complete sample :

import requests import json try: response = requests.get( my_url ) response.raise_for_status() # we want HTTP errors (as 401 Unauthorized) to raise exceptions if response.status_code == 200: json_data = json.loads( response.text ) # parse a JSON string and convert into a dictionary json_string = json.dumps(json_data, indent=4) # convert to a string print ( '+++ (1) JSON [%s].' % json_string ) # ... or ... jsondata = response.json() # use builtin JSON decoder - requests jsonstring = json.dumps( jsondata, indent=4, sort_keys=True ) # convert a python dictionary into a JSON string print ( '+++ (2) JSON [%s].' % jsonstring ) # *** (1) and (2) are identical *** else: print( "--- status not 200 error: ", response.status_code ) except requests.exceptions.HTTPError as errh: # python 3 only print( "--- HTTP error: ", errh ) except requests.exceptions.ConnectionError as errc: print( "--- error connecting: ", errc ) except requests.exceptions.Timeout as errt: print( "--- timeout error: ", errt ) except requests.exceptions.TooManyRedirects: # tell the user their URL was bad and try a different one print( '--- redirect.' ) except requests.exceptions.RequestException as errex: # python 3 only print( '--- request exception:', errex ) raise SystemExit(errex)
Errors and exceptions

In the event of a network problem (e.g. DNS failure, refused connection, etc), Requests will raise a ConnectionError exception

Response.raise_for_status() will raise an HTTPError if the HTTP request returned an unsuccessful status code

If a request times out, a Timeout exception is raised

If a request exceeds the configured number of maximum redirections, a TooManyRedirects exception is raised

All exceptions that Requests explicitly raises inherit from requests.exceptions.RequestException

CURL requests to pvoutput

Parameters to the GET request, as "h=1" : {url} : "URL&h=1"

The GET request expects sid and key while the POST expects X-Pvoutput-SystemId and X-Pvoutput-ApiKey. forum

To set HTTP headres, use "-H" : curl -H "X-Header: value" https://www.keycdn.com

$ curl -H "X-Pvoutput-Apikey: MYKEY" -H "X-Pvoutput-SystemId: MYID" 'https://pvoutput.org/service/r2/getstatus.jsp?&h=1&limit=288&d=20221027'

Otherwise, use sid and key keywords :

$ curl 'https://pvoutput.org/service/r2/getstatus.jsp?sid=MYSID&key=MYKEY&h=1&limit=288&d=20221028' Date Time Ener Efici Pow Ave Normalised gy ency er rage | | | | | | | 20221028,18:15,15796,2.480,120,120,0.019,NaN,NaN,NaN,NaN; 20221028,18:10,15786,2.478,163,168,0.026,NaN,NaN,NaN,NaN; 20221028,18:05,15772,2.476,193,192,0.030,NaN,NaN,NaN,NaN; 20221028,18:00,15756,2.473,296,300,0.047,NaN,NaN,NaN,NaN; 20221028,17:55,15731,2.470,385,384,0.060,NaN,NaN,NaN,NaN; 20221028,17:50,15699,2.465,499,504,0.079,NaN,NaN,NaN,NaN; 20221028,17:45,15657,2.458,556,552,0.087,NaN,NaN,NaN,NaN; 20221028,17:40,15611,2.451,551,552,0.087,NaN,NaN,NaN,NaN; 20221028,17:35,15565,2.443,564,564,0.089,NaN,NaN,NaN,NaN; 20221028,17:30,15518,2.436,642,648,0.102,NaN,NaN,NaN,NaN; 20221028,17:25,15464,2.428,727,732,0.115,NaN,NaN,NaN,NaN; 20221028,17:20,15403,2.418,662,660,0.104,NaN,NaN,NaN,NaN; 20221028,17:15,15348,2.409,670,672,0.105,NaN,NaN,NaN,NaN; 20221028,17:10,15292,2.401,714,720,0.113,NaN,NaN,NaN,NaN;
python requests to pvoutput

Download historical solar photovoltaic data from PVOutput.org : python code


Managing RESTful API

REST stands for "representational state transfer" and is a software architecture style that defines a pattern for client and server communications over a network.

REST defines the following architectural constraints:

realpython -> "Flask", "Django REST Framework", "FastAPI"

{Albert, 20241223} - bottlepy

HTTP methods

REST APIs listen for HTTP methods like GET, POST, and DELETE to know which operations to perform on the web service’s resources.
A resource is any data available in the web service that can be accessed and manipulated with HTTP requests to the REST API.
The HTTP method tells the API which action to perform on the resource.

CRUD operations to HTTP methods :

For example, here’s one of the URLs for GitHub’s REST API :

https://api.github.com/users/<username>

This URL allows you to access information about a specific GitHub user.

Un altre exemple :

$ curl -i http://127.0.0.1:5000/countries \ -X POST \ -H 'Content-Type: application/json' \ -d '{"name":"Germany", "capital": "Berlin", "area": 357022}' HTTP/1.0 201 CREATED Content-Type: application/json ... { "area": 357022, "capital": "Berlin", "id": 4, "name": "Germany" }

This curl command has some options that are helpful to know :

Mind Best practices for REST API security: Authentication and authorization

Un servidor de requeriments REST em sembla que ens empeny directament a nodeJS i Express

REST API server using python

simplest python socket example, server side:

import socket serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) serversocket.bind(('localhost', 8089)) serversocket.listen(5) # become a server socket, maximum 5 connections while True: connection, address = serversocket.accept() buf = connection.recv(64) if len(buf) > 0: print buf break

Or you can use HTTPServer module alongwith SimpleHTTPRequestHandler to create a simple webserver that serves your GET and POST request

python REST API server by Gemini

Input : "write a minimal REST server in python, request is pure text "
This version of the server is designed to receive and process plain text data sent in the body of a POST request.

Output :

sebas@minie:~/dades/python/rest$ cat rest_server.py #!/usr/bin/python3 # -*- coding: utf-8 -*- # install Flask using Mint Software Manager -> python3-flask from flask import Flask, request, Response import logging import traceback app = Flask(__name__) # Configure logging logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s') @app.route('/upload', methods=['POST']) def upload_text(): """ Handles text uploads. The text data is expected in the body of the POST request. """ if request.method == 'POST': # Check if the content type is correct. We expect text/plain if not request.headers['Content-Type'].startswith('text/plain'): logging.error(f"--- Invalid content type: {request.headers['Content-Type']}. Expected text/plain") return Response("Invalid content type. Expected text/plain", status=400) # Check if there is data in the request. if not request.data: logging.error("--- Empty request body") return Response("Empty request body", status=400) try: # Read the text data from the request as a string. text_data = request.data.decode('utf-8') # Decode the byte data to a string logging.info(f"+++ Received text: {text_data}") # Process the text data as needed. Here, we just log it. # You might save it to a file, database, or perform some other action. # Generate a response response_text = f"+++ Text received and processed successfully: {text_data}" return Response(response_text, status=200) except Exception as e: # Log the error with the full traceback. logging.error(f"---Error processing text: {e}\n{traceback.format_exc()}") # Return a 500 Internal Server Error return Response(f"Error processing text: {e}", status=500) else: # If the method is not POST return Response("--- used method is not POST, not allowed", status=405) if __name__ == '__main__': # Start the Flask development server. For production, use a proper WSGI server. app.run(debug=True, host='0.0.0.0', port=5000)

Details :

Content Type Handling: The server checks for Content-Type: text/plain. The error message reflects the expected content type. Data Handling: The code reads the request data as bytes using request.data and then decodes it into a string using .decode('utf-8'). The example processing logs the received text. You can replace this with your actual text processing logic. Response: The server sends a text response, including the received data.

Lets run it :

sebas@minie:~/dades/python/rest$ ./rest_server.py * Serving Flask app 'rest_server' (lazy loading) * Environment: production WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. * Debug mode: on 2025-05-13 17:00:03,688 - WARNING - * Running on all addresses. WARNING: This is a development server. Do not use it in a production deployment. 2025-05-13 17:00:03,688 - INFO - * Running on http://192.168.1.147:5000/ (Press CTRL+C to quit) 2025-05-13 17:00:03,690 - INFO - * Restarting with stat 2025-05-13 17:00:03,785 - WARNING - * Debugger is active! 2025-05-13 17:00:03,838 - INFO - * Debugger PIN: 631-935-736

The client is written in bsh and can be found here

REST API server using C

crec recordar que era ... socket() + bind() + listen() + recv()

C REST API server URL's

REST API docs and URLs

using VPN fom code

Si no vols que tota la màquina passi per la VPN, sinó només el teu programa, pots utilitzar un Proxy SOCKS5. Molts serveis de VPN ofereixen aquesta opció. Pots fer servir una eina anomenada proxychains.

on trobar VPN servers

Estatics :

IP Address P ort País Velocitat Estimada 184.174.68.21 4145 Estats Units Mitjana 98.162.25.7 4145 Estats Units Baixa 72.223.168.67 4145 Estats Units Alta 192.252.209.155 14455 Estats Units Mitjana

On trobar-ne més en temps real?

Com provar-los des de la teva terminal d'Ubuntu?

Abans de posar-lo al teu programa Python, pots verificar si el proxy està viu amb curl:

curl --socks5-hostname 184.174.68.21:4145 https://ifconfig.me

Si et retorna una IP diferent de la teva, el proxy funciona.

proxy local : Tor

Si el teu objectiu és fer scraping de moltes URLs, els proxys gratuïts et donaran molts errors de connexió (ConnectionTimeout). Per a un projecte seriós a Ubuntu 18.04, et sortiria molt més a compte utilitzar Tor com a proxy local:

Tor - configuracio

  1. instal·lem el dimoni de Tor (el motor que corre en segon pla):

    sudo apt update sudo apt install tor -y
  2. Configuració del Proxy (fitxer /etc/tor/torrc [servidor] -> torsocks.conf [client])

    SocksPort 9050 # port a fer servir TorPort SocksListenAddress 127.0.0.1 # TorAddress MaxCircuitDirtiness 300 # canvi de circuit cada 300 seg = 5 min Log notice file /var/log/tor/notices.log
  3. Reiniciar i verificar

    sudo systemctl restart tor # Reinicia el servei netstat -an | grep 9050 # Comprova si el port 9050 està escoltant
  4. Per estar segur que Tor funciona abans de llançar el programa massiu, executa això a la terminal:

    curl --socks5 127.0.0.1:9050 https://check.torproject.org/api/ip

    Si et respon amb un JSON que diu {"IsTor":true, ...}, la teva configuració és perfecta.

  5. Com utilitzar-lo amb el teu programa

    A. Sense tocar el codi (Recomanat):
    Fes servir l'utilitat torsocks. Aquesta eina intercepta les crides de xarxa del teu script i les desvia cap al proxy local de Tor automàticament.

    torsocks python3 el_teu_script.py

    B. Especificant el proxy manualment:
    Si el teu programa permet configurar un proxy, les dades que has d'introduir són:

    Protocol: socks5 IP: 127.0.0.1 Port: 9050
Tor - depuracio

Missatge curios :

At least one protocol listed as required in the consensus is not supported by this version of Tor. You should upgrade. This version of Tor will not work as a client on the Tor network. The missing protocols are: FlowXtrl=1
Tor - actualitzacio

Netegem :

sudo apt-get purge tor sudo apt-get autoremove

Engeguem :

sebastia@t60:~/Downloads/tor-browser_en-US/Browser/TorBrowser/Tor$ ./tor --SocksPort 9050 --DataDirectory ~/.tor_data Feb 17 16:35:55.098 [notice] Tor 0.4.5.10 (git-fd74f7628eba2525) running on Linux with Libevent 2.1.8-stable, OpenSSL 1.1.1, Zlib 1.2.11, Liblzma N/A, Libzstd N/A and Glibc 2.27 as libc. Feb 17 16:35:55.098 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning Feb 17 16:35:55.099 [notice] Configuration file "/var/tmp/dist/tor/etc/tor/torrc" not present, using reasonable defaults. Feb 17 16:35:55.104 [notice] Opening Socks listener on 127.0.0.1:9050 Feb 17 16:35:55.104 [notice] Opened Socks listener connection (ready) on 127.0.0.1:9050 Feb 17 16:35:55.000 [notice] Bootstrapped 0% (starting): Starting Feb 17 16:35:55.000 [notice] Starting with guard context "default" Feb 17 16:35:56.000 [notice] Bootstrapped 5% (conn): Connecting to a relay Feb 17 16:36:02.000 [notice] Bootstrapped 10% (conn_done): Connected to a relay Feb 17 16:36:06.000 [notice] Bootstrapped 14% (handshake): Handshaking with a relay Feb 17 16:36:06.000 [notice] Bootstrapped 15% (handshake_done): Handshake with a relay done Feb 17 16:36:06.000 [notice] Bootstrapped 20% (onehop_create): Establishing an encrypted directory connection Feb 17 16:36:06.000 [notice] Bootstrapped 25% (requesting_status): Asking for networkstatus consensus Feb 17 16:36:06.000 [notice] Bootstrapped 30% (loading_status): Loading networkstatus consensus Feb 17 16:36:10.000 [notice] I learned some more directory information, but not enough to build a circuit: We have no usable consensus. Feb 17 16:36:10.000 [notice] Bootstrapped 40% (loading_keys): Loading authority key certs Feb 17 16:36:13.000 [warn] At least one protocol listed as recommended in the consensus is not supported by this version of Tor. You should upgrade. This version of Tor will eventually stop working as a client on the Tor network. The missing protocols are: FlowCtrl=2 Relay=4 Feb 17 16:36:13.000 [notice] The current consensus has no exit nodes. Tor can only build internal paths, such as paths to onion services. Feb 17 16:36:13.000 [notice] Bootstrapped 45% (requesting_descriptors): Asking for relay descriptors Feb 17 16:36:13.000 [notice] I learned some more directory information, but not enough to build a circuit: We need more microdescriptors: we have 0/9785, and can only build 0% of likely paths. We have 0% of guards bw, 0% of midpoint bw, and 0% of end bw (no exits in consensus, using mid) = 0% of path bw. Feb 17 16:36:13.000 [notice] Bootstrapped 50% (loading_descriptors): Loading relay descriptors Feb 17 16:36:18.000 [notice] The current consensus contains exit nodes. Tor can build exit and internal paths. Feb 17 16:36:30.000 [notice] Bootstrapped 56% (loading_descriptors): Loading relay descriptors Feb 17 16:36:32.000 [notice] Bootstrapped 63% (loading_descriptors): Loading relay descriptors Feb 17 16:36:33.000 [notice] Bootstrapped 69% (loading_descriptors): Loading relay descriptors Feb 17 16:36:35.000 [notice] Bootstrapped 75% (enough_dirinfo): Loaded enough directory info to build circuits Feb 17 16:36:35.000 [notice] Bootstrapped 80% (ap_conn): Connecting to a relay to build circuits Feb 17 16:36:35.000 [notice] Bootstrapped 85% (ap_conn_done): Connected to a relay to build circuits Feb 17 16:36:35.000 [notice] Bootstrapped 89% (ap_handshake): Finishing handshake with a relay to build circuits Feb 17 16:36:35.000 [notice] Bootstrapped 90% (ap_handshake_done): Handshake finished with a relay to build circuits Feb 17 16:36:35.000 [notice] Bootstrapped 95% (circuit_create): Establishing a Tor circuit Feb 17 16:36:36.000 [notice] Bootstrapped 100% (done): Done

T60 with Ubuntu 18.04.4 is too old to run latest Tor.


Managing CSV

Sample :

pi@R4:~/python/pkw/eines/factura $ cat 2_csv_test.py #!/usr/bin/env python # -*- coding: UTF-8 -*- import sys pfnCSV = 'csv_test.csv' # formated lines as "00:02,0,-316,316" def leer_csv ( fnCSV ) : n = 0 fdIN = open( fnCSV, 'r' ) for line in fdIN: n = n + 1 line = line.strip() # remove trailing \n (key, PP, PC, PD) = line.split(",") # split CSV using "," separator print ( "CSV ({}) {} | {} | {} | {}".format (n, key, PP, PC, PD) ) fdIN.close() if __name__ == "__main__": leer_csv ( pfnCSV ) sys.exit()

Also pi@R4:~/python/pkw/eines/veure_potencia_instant/5_verificar_fitxer_csv.py and pi@R4:~/python/pkw/eines/factura/1_factura.py

Input and output :

pi@R4:~/python/pkw/eines/factura $ more csv_test.csv 00:00,0,-322,322 00:01,0,-329,329 00:02,0,-316,316 00:03,0,-398,398 pi@R4:~/python/pkw/eines/factura $ ./2_csv_test.py CSV (1) 00:00 | 0 | -322 | 322 CSV (2) 00:01 | 0 | -329 | 329 CSV (3) 00:02 | 0 | -316 | 316 CSV (4) 00:03 | 0 | -398 | 398
python read JSON data from CSV file

Use CSV module, specifically

class csv.DictReader ( f, fieldnames=None, restkey=None, restval=None, dialect='excel', *args, **kwds )

Create an object that operates like a regular reader but maps the information in each row to a dict whose keys are given by the optional fieldnames parameter.

Trick : if fieldnames is omitted, the values in the first row of file f will be used as the fieldnames.

python examples - compte amb el format

els noms dels camps son a 1a linia si no es defineixen explicitament !

my_fieldnames = ['date', 'PP', 'PC', 'PD'] with open(csvFilePath, 'r') as csvf: csvReader = csv.DictReader(csvf, fieldnames=my_fieldnames) # load csv file data into a dictionary r

python and JSON

JSON can store lists, bools, numbers, tuples and dictionaries.
But to be saved into a file, all these structures must be reduced to strings, so they can be read or written to a file.

Sample :

import json response = requests.get(url, timeout=60) response.raise_for_status() if response.status_code == 200: json_data = json.loads(response.text) # parse a JSON string and convert into a dictionary json_string = json.dumps(json_data, indent=4) # convert to a string print ( '+++ (2) JSON [%s].' % json_string ) else: print( "--- status not 200 error: ", response.status_code )

See complete doc

python json.load(s) sample

load() and loads() :

import json with open("developer.json", "r") as read_file: # read JSON file developer = json.load(read_file) # convert JSON encoded data into python dictionary for key, value in developer.items(): # display decoded JSON data from file print(key, ":", value)

A mi m'agrada sense "with" :

import json f = open('data.json') # open JSON file dict = json.load(f) # get JSON object as a dictionary *** does NOT keep order in file *** f.close() # close file for key, value in dict.items(): # display JSON data print(key, ":", value)
python json.dump(s) details

dump() and dumps() :

Serialize obj to a JSON formatted string using this conversion table.

json.dumps( obj, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=None, indent=None, separators=None, default=None, sort_keys=False, **kw )
python displays JSON data with nice format

import json url = "http://" + hostname + endpoint r = requests.get(url, timeout=60) r.raise_for_status() jsondata = r.json() print( json.dumps( jsondata, indent=4, sort_keys=True ) )
python writes/reads JSON data to a file with nice format

hostname = "fronius" version = "1.8.1-9" def get_example(endpoint): print("Fetching " + endpoint) url = "http://" + hostname + endpoint r = requests.get(url, timeout=60) r.raise_for_status() jsondata = r.json() m = re.match(r".*/(.*)\..*", endpoint) filename = m.group(1) directory = os.path.join("examples", "json", version) if not os.path.exists(directory): os.makedirs(directory) with open(os.path.join(directory, filename + ".json"), 'w') as f: # output file = GetAPIVersion.json json.dump(jsondata, f, indent=4, sort_keys=True) def main(argv): get_example("/solar_api/GetAPIVersion.cgi") if __name__ == "__main__": main(sys.argv)
python read/write JSON data from file

The JSON data has this format :

{ "qq": 0, "ts_ini": "2022-07-25 09:30:04", "ts_fin": "2022-07-24 11:00:04" }

And we manage it like this :

#!/usr/bin/env python3 import datetime import json import os import sys myFN = "jb_json.txt" def My_Main(): try: pf = open( myFN, 'r' ) mj = json.load( pf ) # read json from file into dictionary print( mj ) mq = mj['qq'] mt = mj['ts'] print( "Q = ", mq ) print( "TS = ", mt ) pf.close() pf = open( myFN, 'w' ) mj['qq'] = 0 mj['ts'] = "12:12:12, 02/03/2012" json.dump( mj, pf ) # write json to file pf.close() except: print( "--- Unexpected error: ", str( sys.exc_info()[0] ) ) if __name__ == '__main__': My_Main() sys.exit() # bye
python plot graph using JSON files

educative.io : How to plot graphs using .json files in Python
Easy "line" graph and easy "bar" graph

consells de'n Albert per fer una grafica amb python


matplotlib

Que es ? plotting library
Matplotlib along with NumPy can be treated as the open source equivalent of MATLAB.

Tutorial, examples, nice guide, documentacio 3.9.0

instalacio

matplotlib.org

Just as we use the np shorthand for NumPy and the pd shorthand for Pandas, we will use some standard shorthands for Matplotlib imports:

import matplotlib as mpl import matplotlib.pyplot as plt

The plt interface is what we will use most often.

O'Reilly "Python Data Science Handbook" by Jake VanderPlas - Chapter 4. Visualization with Matplotlib {****}

versio

$ python3 -c 'import matplotlib; print(matplotlib.__version__, matplotlib.__file__)' import matplotlib print( "Version ({})".format(matplotlib.__version__) )

I a les meves maquines :

sebas@minie:~/dades/_local_tinet_files$ python3 -c 'import matplotlib; print(matplotlib.__version__, matplotlib.__file__)' 3.5.1 /usr/lib/python3/dist-packages/matplotlib/__init__.py pi@pi0:~/njs/fronius $ python3 -c 'import matplotlib; print(matplotlib.__version__, matplotlib.__file__)' 3.5.2 /home/pi/.local/lib/python3.7/site-packages/matplotlib/__init__.py pi@pi0:~/njs/fronius $ python2 -c 'import matplotlib; print(matplotlib.__version__, matplotlib.__file__)'
Instalacio

Si en executar un python ens trobem amb

ModuleNotFoundError: No module named 'matplotlib'

... cal instalar "matplotlib"

Minie 20240520 - version 3.5.1-2build1

Pi0 :

pi@pi0:~/njs/fronius $ sudo apt-get install python-matplotlib pi@pi0:~/njs/fronius $ python2 -c 'import matplotlib; print(matplotlib.__version__, matplotlib.__file__)' ('2.2.3', '/usr/lib/python2.7/dist-packages/matplotlib/__init__.pyc')
Conceptes : APIs i names of objects

Matplotlib is strongly object oriented and its principal objects are the figure and the axes.

You can think of the figure as a canvas, of which you typically specify the dimensions and possibly e.g., the background color etc etc.
You use the canvas, the figure, essentially in two ways, placing other objects on it (mostly axes, but also text labels etc) and saving its contents with savefig.

You can think of an axes as a sort of Swiss Army knife, a handy object that offers a tool (e.g. .plot, .scatter, .hist etc) for everything, mostly.
You can place one, two, ... many axes inside a figure using one of many different methods.

stackoverflow

Read The Matplotlib Object Hierarchy

Matplotlib has two interfaces - API reference

... an Axes actually translates into what we think of as an individual plot or graph (rather than the plural of “axis”)

Two interfaces for the price of one

A potentially confusing feature of Matplotlib is its dual interfaces: a convenient MATLAB-style state-based interface, and a more powerful object-oriented interface.

MATLAB-style interface

Matplotlib was originally written as a Python alternative for MATLAB users, and much of its syntax reflects that fact. The MATLAB-style tools are contained in the pyplot (plt) interface.

It’s important to note that this interface is stateful: it keeps track of the “current” figure and axes, which are where all plt commands are applied.
You can get a reference to these using the plt.gcf() (get current figure) and plt.gca() (get current axes) routines.

plt.figure() # create a plot figure # create the first of two panels and set current axis plt.subplot(2, 1, 1) # (rows, columns, panel number) plt.plot(x, np.sin(x)) # create the second panel and set current axis plt.subplot(2, 1, 2) plt.plot(x, np.cos(x));
Object-oriented interface

The object-oriented interface is available for these more complicated situations, and for when you want more control over your figure. Rather than depending on some notion of an “active” figure or axes, in the object-oriented interface the plotting functions are methods of explicit Figure and Axes objects.

# First create a grid of plots # ax will be an array of two Axes objects fig, ax = plt.subplots(2) # Call plot() method on the appropriate object ax[0].plot(x, np.sin(x)) ax[1].plot(x, np.cos(x))

O'Reilly

input file format

Per fer el parsing de python mes facil, lo millor es guardar les dades en format CSV

pi@R4:~/python/pkw/dades/potencies/csv $ cat pot_CSV_2024_07_19.csv 10:11,2617,-82,-2534
un exemple senzill

{ see /home/sebas/dades/python/matplot/4_mpl.py } def plot_csv ( fnCSV, fnPNG ) : if beVerbose : print( ">>> csv in : CSV file {%s}" % fnCSV ) print( ">>> png out : PNG file {%s}" % fnPNG ) fig = plt.figure( figsize=( 1800/my_dpi, 800/my_dpi ), dpi=my_dpi ) # volem PNG de 1800x800 pixels ax = fig.subplots( ) # "fig" is the canvas, "ax" is the graph ax.set_facecolor("gray") # ax.set_xticks( [0, 60, 120, 180, 240, 300, 360, 420, 480, 540, 600, 660, 720, 780, 840, 900, 960, 1020, 1080, 1140, 1200, 1260, 1320, 1380, 1440], # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24] ) ax.set_xticks( [0, 60, 120, 180, 240, 300, 360, 420, 480, 540, 600, 660, 720, 780, 840, 900, 960, 1020, 1080, 1140, 1200, 1260, 1320, 1380, 1440] ) ; ax.set_xticklabels( [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24 ] ) X = [] Y = [] n = 0 with open( fnCSV, 'r') as datafile: plotting = csv.reader(datafile, delimiter=',') # timestamp , potencia , etc , etc for ROWS in plotting: n = n + 1 X.append( n ) Y.append( int(ROWS[1]) ) ax.plot( X, Y, color='red', label = 'pProd' ) # specify line colour and graph label ax.legend() # display label ! ax.fill_between( X, Y, facecolor='green' ) # fill_between uses the colors of the color cycle as the fill color ax.set_xlabel('eix X (h)') ax.set_ylabel('eix Y (kWh)', fontweight='bold') # https://www.geeksforgeeks.org/how-to-set-plot-background-color-in-matplotlib/ now = datetime.datetime.now() szTimestamp = now.strftime("%Y-%m-%d %H:%M:%S %Z") szSubtitle = szTimestamp + '- CSV line graph, versio {' + szVersion + '}' plt.suptitle( szSubtitle ) plt.title('dades : {%s}' % fnCSV ) print( "+++ graph in file {%s}" % fnPNG ) plt.savefig( fqfn_fnG ) return 0
funcions clau

Paraules clau :

.figure()

Permet especificar el tamany del PNG - width, height

Per defecte, treu un PNG de 640 x 480 pixels, i te per defecte DPI=100, aixi que es codifica figsize=(6.4, 4.8) per defecte

# plt.figure(figsize=(3.20, 2.40)) # DPI = 100 -> PNG de 320 x 240 pixels

Matplotlib doesn't work with pixels directly, but rather physical sizes and DPI. If you want to display a figure with a certain pixel size, you need to know the DPI of your monitor.

For example this link will detect that for you.

Volem produir un PNG de un tamany determinat, diguem 1800 x 800 pixels :

my_dpi = 90 plt.figure( figsize=( 1800/my_dpi, 800/my_dpi ), dpi=my_dpi )

stackoverflow

my_dpi = 96x96 {MARS}, 264x264 {Xiaomi 10}, 144x144 {tablet "Tab A8"}

.plot()

Plot y versus x as lines and/or markers.

.show()

matplotlib will simply output the chart to your viewport when the .show() method is invoked

plt.show() ./z1.py:17: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure. you pip install matplotlib and you are trying to display a plot in a GUI window and you do not have a python module for GUI display.
how to display under GUI

stackoverflow

Use "spider", python IDE for scientists :

$ sudo apt install python3 python3-matplotlib spyder3 ipython3 # Python 3.x, askubuntu
.savefig()

Save the current figure as an image or vector graphic to a file.

Amb aquesta ajuda ho treiem a fitxer !

#!/usr/bin/env python # -*- coding: UTF-8 -*- import matplotlib.pyplot as plt data = {'apple': 10, 'orange': 15, 'lemon': 5, 'lime': 20} <<<< la entrada es un diccionari de python names = list(data.keys()) values = list(data.values()) fig, axs = plt.subplots(1, 3, figsize=(9, 3), sharey=True) axs[0].bar(names, values) axs[1].scatter(names, values) axs[2].plot(names, values) fig.suptitle('Categorical Plotting') # plt.show() # mostrar a pantalla ... plt.savefig('books_read.png') # ... o guardar a fitxer

The available output formats depend on the backend being used. Onminie, if I run the code

fig, ax = plt.subplots() sz=fig.canvas.get_supported_filetypes() print( sz )

... we get {20250105}

{'eps': 'Encapsulated Postscript', 'jpg': 'Joint Photographic Experts Group', 'jpeg': 'Joint Photographic Experts Group', 'pdf': 'Portable Document Format', 'pgf': 'PGF code for LaTeX', 'png': 'Portable Network Graphics', 'ps': 'Postscript', 'raw': 'Raw RGBA bitmap', 'rgba': 'Raw RGBA bitmap', 'svg': 'Scalable Vector Graphics', 'svgz': 'Scalable Vector Graphics', 'tif': 'Tagged Image File Format', 'tiff': 'Tagged Image File Format'}
.ylim()

It is used for setting the y-axis range (or to read it)

Es a dir, si tenim dades en el rang [38..52], per defecte el "38" sera el començament del eix "Y".
Si volem que comenci a "0", hem de posar .ylim()

Atencio : ha de estar ben aprop de .plot() - mai abans de .figure()

.subplots()

Add an Axes to the current figure or retrieve an existing Axes.

The default call is subplots(nrows=1, ncols=1).

nice article, as Text inside dollar signs utilizes TeX markup to put variables in italics !!!

capçaleres

Sempre se li ha de dir

escriu un programa python, començant per #!/usr/bin/python3 # -*- coding: UTF-8 -*- que faci ...
styles

Creant codi amb Gemini, vols fer servir

OSError: 'seaborn-v0_8-whitegrid' not found in the style library and input is not a valid URL or path; see `style.available` for list of available styles

Mirem quins estils tenim :

sebas@minie:~/dades/python/matplot$ cat display_available_styles.py #!/usr/bin/python3 # -*- coding: UTF-8 -*- import matplotlib.pyplot as plt # List all available styles styles = plt.style.available print(styles)

I el resultat es :

sebas@minie:~/dades/python/matplot$ ./display_available_styles.py ['Solarize_Light2', '_classic_test_patch', '_mpl-gallery', '_mpl-gallery-nogrid', 'bmh', 'classic', 'dark_background', 'fast', 'fivethirtyeight', 'ggplot', 'grayscale', 'seaborn', 'seaborn-bright', 'seaborn-colorblind', 'seaborn-dark', 'seaborn-dark-palette', 'seaborn-darkgrid', 'seaborn-deep', 'seaborn-muted', 'seaborn-notebook', 'seaborn-paper', 'seaborn-pastel', 'seaborn-poster', 'seaborn-talk', 'seaborn-ticks', 'seaborn-white', 'seaborn-whitegrid', 'tableau-colorblind10']
proves

Fem proves a MARS - obtenim PNG de 640 x 480 pixels

/home/nicolau/sebas/python/matplot/2_plot.py -> ./line_graph.png

I amb dades reals treballem a

(r0) /home/pi/python/fronius/z_extract_temp_from_json_and_do_graph.py -> ./d_temperatures/line_graph_2022-08-04.png

Compte : al PiZero hem de fer servir python3, vist que no es pot instalar matplotlib per python2

R4 : llegim texte temperatures i generem PNG :

pi@R4:~/python/termometre $ cat 2_plot_temp.py # every line is in this format : # 09:45:02 - T 193 , H 52 # 0 1 2 3 for line in open(inFN, 'r'): lines = [i for i in line.split()] print("0 = {%s}" % lines[0]) print("3 = {%s}" % lines[3]) x.append(lines[0]) y.append(int(lines[3])) plt.yticks(y) plt.plot(x, y, marker = 'o', c = 'g') # plt.plot() # plt.show() plt.savefig(outFN)
backend

To support all use cases, matplotlib can target different outputs, and each of these capabilities is called a backend; the "frontend" is the user facing code, i.e., the plotting code, whereas the "backend" does all the hard work behind-the-scenes to make the figure.

Matplotlib backend relies on some backend to actually render the plots. The default backend is the agg backend. This backend only renders PNGs. Docs for Matplotlib backend.

So, if you want to generate an image but not to display it, use this code :

import matplotlib matplotlib.use('Agg') import matplotlib.pyplot as plt … plt.savefig(outFN)
matplotlib problems

Jul 21 18:34:56 pi0 fronius-nodejs[10603]: *** 2024/07/21, 18:34:56 - --- python POT RC (Error: /home/pi/njs/fronius/5_potencia_csv_a_png.py:45: UserWarning: Jul 21 18:34:56 pi0 fronius-nodejs[10603]: This call to matplotlib.use() has no effect because the backend has already Jul 21 18:34:56 pi0 fronius-nodejs[10603]: been chosen; matplotlib.use() must be called *before* pylab, matplotlib.pyplot, Jul 21 18:34:56 pi0 fronius-nodejs[10603]: or matplotlib.backends is imported for the first time. Jul 21 18:34:56 pi0 fronius-nodejs[10603]: The backend was *originally* set to 'TkAgg' by the following code: Jul 21 18:34:56 pi0 fronius-nodejs[10603]: File "/home/pi/njs/fronius/5_potencia_csv_a_png.py", line 42, in <module> Jul 21 18:34:56 pi0 fronius-nodejs[10603]: import matplotlib.pyplot as plt

Solucio :

pi@pi0:~/njs/fronius $ cat 5_potencia_csv_a_png.py import matplotlib # 20240721 matplotlib.use('Agg') # 20240721, Error: _tkinter.TclError: no display name and no $DISPLAY environment variable # matplotlib.use() must be called *before* pylab, matplotlib.pyplot, # ... or matplotlib.backends is imported for the first time. import matplotlib.pyplot as plt #
URLs matplotlib


my python projects


exercise : get sys info into text/html file + ftp

Recollir informacio del sistema, escriure un fitxer HTML i enviar-lo per FTP

Resultat : PiZero , Odin , by //MARS:/home/nicolau/sebas/python/tinet/R0/enviar_ip_a_tinet.py


Send message to Telegram bot

Enviar texte a un bot - tg.htm


Manage Raspberry GPIO using python

What is RPi.GPIO?
It’s an easy way of controlling the Pi’s General Purpose Input Output ports in python.

Display RPi.GPIO version

#!/usr/bin/env python3 import RPi.GPIO as GPIO # sudo apt-get install python3-rpi.gpio print( "GPIO versio ", GPIO.VERSION ) # mostrar la versio de GPIO print( "RPi revision ", GPIO.RPI_REVISION ) # mostrar la revisio del Raspberry Pi - url
Basic GPIO code

sag@odin:~/gpio_python $ cat bucle_sortida_pin37.py #!/usr/bin/env python3 # -*- coding: utf-8 -*- # executem amb "sudo ./bucle_sortida_pin37.py" import RPi.GPIO as GPIO import time GPIO.setmode(GPIO.BOARD) # escollim numeracio "com a la placa" SORTIDA = 37 # com sortida fem servir el penultim pin de l'esquerra (see command "pinout") - GPIO26, pin 37 GPIO.setup( SORTIDA, GPIO.OUT ) # programem el pin com de "sortida" i = 0 retard = 2 while i < 100: # white True: i = i + 1 print( 'i = ', i ) # mostrem activitat print( '(1) encenem el llum' ) GPIO.output(SORTIDA, GPIO.HIGH) # output "ON" print( '(2) esperem (%s) sg' % retard ) time.sleep( retard ) # wait(2 sg) print( '(3) apaguem el llum' ) GPIO.output(SORTIDA, GPIO.LOW) # output "OFF" print( '(4) esperem (%s) sg' % retard ) time.sleep( retard ) # wait(2 sg) print( '(99) netejem i acabem' ) GPIO.cleanup()
"channel in use" warning

If we get

RuntimeWarning: This channel is already in use, continuing anyway. Use GPIO.setwarnings(False) to disable warnings.

We have to improve our code to execute "cleanup()" on all cases :

retard = 2 # nombre de segons q esperem try: while True: # forever loop ; exit using SIGINT i = i + 1 print( 'i = ', i ) # mostrem activitat print( '(1) encenem el llum' ) GPIO.output(SORTIDA, GPIO.HIGH) print( '(2) esperem (%i) segons' % retard ) time.sleep( retard ) # wait() print( '(3) apaguem el llum' ) GPIO.output(SORTIDA, GPIO.LOW) print( '(4) esperem (%s) segons' % retard ) time.sleep( retard ) except KeyboardInterrupt: print( '--- tenim CTRL C' ) except: print( '--- --- ALL other exceptions' ) finally: print( '>>> netejem i acabem' ) GPIO.cleanup()

Read raspi.tv


web server
run one-liner (python3)

From python v3 we can go one-liner :

$ python3 -m http.server 8084 # files are served from current directory $ python3 -m http.server 8084 --directory /tmp/prova/ # from python v 3.7 up

afternerd

Per exemple, si volem mirar un codi HTML + JavaScript local, (MARS+njs+pere) obtindrem un missatge a la consola com :

Access to fetch at 'file:///home/nicolau/sebas/njs/pere/info.html' from origin 'null' has been blocked by CORS policy: Cross origin requests are only supported for protocol schemes: http, data, chrome, chrome-extension, chrome-untrusted, https.

Then, we start a python server and proceed :

nicolau@mars:~/sebas/njs/pere$ python3 -m http.server Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ... 127.0.0.1 - - [14/Aug/2022 09:06:32] "GET / HTTP/1.1" 304 - 127.0.0.1 - - [14/Aug/2022 09:06:32] "GET /styles.css HTTP/1.1" 304 - 127.0.0.1 - - [14/Aug/2022 09:06:32] "GET /main.js HTTP/1.1" 304 -
from code

Volem un web server lleugerament semblant a un node, es a dir que

We can code :

import http.server # docs import socketserver # docs PORT = 8083 Handler = http.server.SimpleHTTPRequestHandler with socketserver.TCPServer(("", PORT), Handler) as httpd: print("serving at port", PORT) httpd.serve_forever()
cgi-bin python web server

We code :

sag@odin:~/python/web_server $ cat /home/sag/python/web_server/cgi/my_cgi_server.py #!/usr/bin/env python3 # web server # run with "./my_cgi_server.py" # sudo adduser sebas gpio # access using http://myraspiodin.hopto.org:8123/cgi-bin/one.py # https://stackoverflow.com/questions/13065306/how-to-execute-python-script-on-the-basehttpserver-created-by-python from http.server import CGIHTTPRequestHandler, HTTPServer my_Host = '192.168.1.123' my_Port = 8123 print( "*** web server, ip (", my_Host, "), port (", my_Port, ")." ) handler = CGIHTTPRequestHandler handler.cgi_directories = ['/cgi-bin', '/htbin'] # this is the default server = HTTPServer(( my_Host, my_Port ), handler) server.serve_forever()

To start, we serve this file :

sag@odin:~/python/web_server/cgi $ cat index.htm <!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <title> *** mon Index *** </title> </head> <body> <hr> <p> <a href="http://myraspiodin.hopto.org:8123/cgi-bin/one.py"> Encendre </a> <p> . <p> << href="http://myraspiodin.hopto.org:8123/cgi-bin/two.py"> Apagar </a> <r> </body> </html>

And the action is performed here :

sag@odin:~/python/web_server/cgi/cgi-bin $ cat one.py #!/usr/bin/env python3 import RPi.GPIO as GPIO GPIO.setmode(GPIO.BOARD) # escollim numeracio "com a la placa" SORTIDA = 37 # com sortida fem servir el penultim pin de l'esquerra (see command "pinout") - GPIO26, pin 37 GPIO.setup( SORTIDA, GPIO.OUT ) # programem el pin com de "sortida" GPIO.output(SORTIDA, GPIO.HIGH) # set pin HIGH print( 'Content-type: text/html\n') print( '<title> +++ ON +++ </title>') print( ' <body> <hr> ' ) print( ' <p> +++ now LED is ON +++ ' ) print( ' <p> Per apagar <a href="http://myraspiodin.hopto.org:8123/cgi-bin/two.py"> pica aqui </a> </p> ' ) print( ' <hr> </body>')

And here :

sag@odin:~/python/web_server/cgi/cgi-bin $ cat two.py #!/usr/bin/env python3 import RPi.GPIO as GPIO GPIO.setmode(GPIO.BOARD) # escollim numeracio "com a la placa" SORTIDA = 37 # com sortida fem servir el penultim pin de l'esquerra (see command "pinout") - GPIO26, pin 37 GPIO.setup( SORTIDA, GPIO.OUT ) # programem el pin com de "sortida" GPIO.output(SORTIDA, GPIO.LOW) # set pin LOW print('Content-type: text/html\n') print('<title> --- OFF --- </title>') print( ' <body> <hr> ' ) print( ' <p> --- now LED is OFF --- ' ) print( ' <p> Per encendre <a href="http://myraspiodin.hopto.org:8123/cgi-bin/one.py"> pica aqui </a> </p> ' ) print( ' <hr> </body>')
python webserver channel warning

We get this interesting runtime warning :

/home/sag/python/web_server/cgi/cgi-bin/one.py:6: RuntimeWarning: This channel is already in use, continuing anyway. Use GPIO.setwarnings(False) to disable warnings. GPIO.setup( SORTIDA, GPIO.OUT ) # programem el pin com de "sortida"

Solution : code cleanup() as indicated here

python web server links

Some URLs :


using Google Sheets from python

  1. login to Google account {pocaporpere}

  2. set up a Google Cloud platform project :

    1. enter Google Cloud console
    2. "Select a project" + "New project" {top right}
    3. Projet Name := "minie python 2 Gdatasheets" + "Create"

  3. enable Google Sheets API :

    1. search for "google sheets api" in top center box
    2. first entry is "Google Sheets API" + "Google Enterprise API" - click on it
    3. "Google Sheets API" page comes up - click "Enable"

  4. no enable Cloud Natural Language API :

    1. search for "cloud natural language api" in top center box
    2. click "Enable"
    3. Cloud Natural Language API requires a project with a billing account

  5. create API key :

    1. open top-left 3-lines and select "APIs and services", then "Credentials"

    2. click "create credentials" - here we have 3 options :

      • API key -
      • OAuth client ID -
      • Service account - get JSON file

    3. + "API key" - it is used to authenticate to the 2 API's
      With this authentication our API will only work with "public" Google Sheets
      To use "private" Google Sheets we'd need to use OAuth authentication (or Service Account ?)

      save the key for later use

    4. access using Google Service Account (JSON file)

      • go to Google Cloud Console

      • ensure the correct project is selected from the dropdown menu at the top of the page

      • navigate to Service Accounts in the navigation menu (the hamburger icon), go to IAM & Admin > Service Accounts

      • click "+ Create Service Account" nom "servei-isins"

      • create the JSON key :

        • Once you are on the Service Account details page, click the Keys tab.
        • Click the Add Key dropdown menu, and select Create new key.
        • In the dialog box that appears, select the JSON key type.
        • Click Create.

      • download the file {20251215 - isin2gss-df6e78707986.json}

  6. Service Account is shared with the Sheet:

    Find the Client Email in your isin2gss-df6e78707986.json file (it looks like [account-name]@[project-id].iam.gserviceaccount.com).

    Go to your Google Spreadsheet, click the Share button, and share the spreadsheet with that specific Service Account email, giving it Editor permission.

  7. enable Google API : visit here

  8. to connect to Google data, we'll be using Google libraries, built in toolkits

    $ pip3 install google-api-python-client google-cloud-language

    Jo faig servir "software manager" + "python3-googleapi" v 1.7.11-4 & python3-googleapi-samples
    destinacio a minie : /usr/share/doc/python3-googleapi-samples/examples
    Homepage

  9. create a Spreadsheet, and name it "miniepython2GS" :

    Here is the private spreadsheet URL and ID is "1W070QyiBSOP_bEwCIyO2wziAYoAV3F4z5QgeIkZq5E0", owned by "pocaporpere"
    Public URL

Using AI we get this code : chatGPT write GSS -> sebas@minie:~/dades/python/gugel_sheets$./4_chat_gpt.py & details (install, errors, ...)

Basic concepts

To write to a sheet, you need the spreadsheet ID, the range of cells in A1 notation, and the data you wish to write within an appropriate request body object.
Updates require a valid ValueInputOption parameter :

Single quotes are required for sheet names with spaces, special characters, or an alphanumeric combination.

Cell

The "Cells" resource represents each cell, but it doesn't have a unique ID value. Instead, row and column coordinates identify the cells.

API overview, concepts

Basic code

developers.google

read cell values

Simple read :

sebas@minie:~/dades/python/gugel_sheets$ cat 2_rw_gss.py from google.oauth2 import service_account # "pip3 install google" + "pip3 install --upgrade google-api-python-client" from googleapiclient.discovery import build spreadsheet_id = "1B987Qyi333P_b647IyOquiquioAV3F4z5QgeIkZq4Ez" credentials = service_account.Credentials.from_service_account_file("key.json", scopes=["https://www.googleapis.com/auth/spreadsheets"]) # set up credentials service = build("sheets", "v4", credentials=credentials) # create Sheets API client request = service.spreadsheets().get(spreadsheetId=spreadsheet_id, ranges=[], includeGridData=False) sheet_props = request.execute() print(sheet_props["properties"]["title"])

hackernoon

update() cell values

Basic write :

sebas@minie:~/dades/python/gugel_sheets$ cat 2_rw_gss_pere.py from google.oauth2 import service_account # "pip3 install google" + "pip3 install --upgrade google-api-python-client" from googleapiclient.discovery import build spreadsheet_id = "1B987Qyi333P_b647IyOquiquioAV3F4z5QgeIkZq4Ez" credentials = service_account.Credentials.from_service_account_file("key.json", scopes=["https://www.googleapis.com/auth/spreadsheets"]) # set up credentials service = build("sheets", "v4", credentials=credentials) # create Sheets API client now = datetime.datetime.now() szTemps = now.strftime("%Y-%m-%d %H:%M:%S") range_name = "Sheet1!B2:B2" # starting and ending cell coordinates new_values = [[szTemps]] # Values are intended to be a 2d array body = {"values": new_values} result = service.spreadsheets().values().update( spreadsheetId=spreadsheet_id, range=range_name, valueInputOption="USER_ENTERED", body=body).execute() range_name = "Sheet1!B4:C6" # starting and ending cell coordinates new_values = [['Name', 'Age'], ['John', '30'], ['Jane', '25']] body = {"values": new_values} # values are intended to be a 2d array result = service.spreadsheets().values().update( spreadsheetId=spreadsheet_id, range=range_name, valueInputOption="USER_ENTERED", body=body).execute()

coefficient.io

If successful, the response body contains an instance of UpdateValuesResponse

com fer "public" un Google SpreadSheet

To make a document, spreadsheet, or presentation available for a large audience to see, publish the file.
After you publish your file you can send a new URL to anyone or embed into your website.

Any changes you make to the original document will be updated in the published version.

  1. go Sheets and login()
  2. at the top left, click "File" and then "Share" and then "Publish to web"
  3. copy "public" sheet URL : Fronius from R4 or xatGPT

support.google.com

la meva Google SpreadSheet amb la potencia diaria del Fronius

El R4 escriu cada nit a aquesta pagina :

write to GSS using chatGPT code

My question was "acces to google spreadsheets from python"

  1. instalem el modul :

    sebas@minie:~/dades/python/gugel_sheets$ python3 -m pip install gspread Defaulting to user installation because normal site-packages is not writeable Collecting gspread Downloading gspread-6.1.4-py3-none-any.whl (57 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.6/57.6 KB 49.7 kB/s eta 0:00:00 Collecting google-auth-oauthlib>=0.4.1 Downloading google_auth_oauthlib-1.2.1-py2.py3-none-any.whl (24 kB) Collecting google-auth>=1.12.0 Downloading google_auth-2.38.0-py2.py3-none-any.whl (210 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 210.8/210.8 KB 59.8 kB/s eta 0:00:00 Requirement already satisfied: rsa<5,>=3.1.4 in /usr/lib/python3/dist-packages (from google-auth>=1.12.0->gspread) (4.8) Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/lib/python3/dist-packages (from google-auth>=1.12.0->gspread) (0.2.1) Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/lib/python3/dist-packages (from google-auth>=1.12.0->gspread) (5.0.0) Collecting requests-oauthlib>=0.7.0 Downloading requests_oauthlib-2.0.0-py2.py3-none-any.whl (24 kB) Requirement already satisfied: requests>=2.0.0 in /usr/lib/python3/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib>=0.4.1->gspread) (2.25.1) Requirement already satisfied: oauthlib>=3.0.0 in /usr/lib/python3/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib>=0.4.1->gspread) (3.2.0) Installing collected packages: requests-oauthlib, google-auth, google-auth-oauthlib, gspread Successfully installed google-auth-2.38.0 google-auth-oauthlib-1.2.1 gspread-6.1.4 requests-oauthlib-2.0.0

  2. error classic :

    gspread.exceptions.SpreadsheetNotFound: <Response [200]>

    Solucio :
    you need to share the spreadsheet with the "client_email" mail id
    open the spreadsheet and then click on the "share" button (top right) and then just paste the "client_email" (email id) from JSON

    stackOverflow, gspread Docs on "Auth"

    Posem "comptexatgpt@xatgpt.iam.gserviceaccount.com" i piquem "Compartir"

  3. error curios :

    gspread.exceptions.APIError: APIError: [400]: Invalid value at 'data.values[0]' (type.googleapis.com/google.protobuf.ListValue), "13:05:34 al cap de munt de tot"

    ValueRange :

    values[] array (ListValue format) The data that was read or to be written. This is an array of arrays, the outer array representing all the data and each inner array representing a major dimension. Each item in the inner array corresponds with one cell.

    Solucio - per "append_row()", el texte ha de ser un array :

    szTexte = [szHMS + " al cap de munt de tot"]

  4. error similar :

    gspread.exceptions.APIError: APIError: [400]: Invalid values[1][0]: list_value { values { string_value: "txt a lloc determinat" } }

    Solucio - per "update_cell()" el texte ha de ser un string

detalls de gspread

gspread is a Python API for Google Sheets - under the hood, gspread uses Google Sheets API v4

gspread homepage (exemples), github (exemples)

Sample :

import gspread # pip install gspread gc = gspread.service_account() # authenticate using a service account worksheet_list = gc.worksheets() # get a list of all worksheets wks = gc.open("Where is the money ?").sheet1 # open a worksheet from a spreadsheet wks.update([[1, 2], [3, 4]], "A1") # update a range of cells using the top left corner address wks.update_acell("B42", "it's down there somewhere.") # update a single cell wks.format('A1:B1', {'textFormat': {'bold': True}}) # format the header
codi python amb gspread

Minie escriu on-demand a aquesta pagina :

I el accés es fa així :

sebas@minie:~/dades/python/gugel_sheets$ cat 4_chat_gpt_ignasi.py
ISIN i python

An ISIN is a permanent, unique 12-character code used to identify a specific security, such as a stock, bond, or fund.

What is the value (e.g., stock price) for a security today?
You would typically use a Ticker Symbol for this in python.
A Ticker Symbol is specific to a single exchange.

To get the stock price (value) using python, you would typically use a library like yfinance or pandas_datareader.
Most free and popular Python libraries like yfinance only accept the Ticker Symbol (e.g., AAPL or QDVW.DE), not the ISIN directly.
Reading the current stock price (value) of an ISIN using Python requires a crucial intermediary step: Symbology Mapping.
The most robust way is by using a dedicated financial API. Two good options that support ISIN lookups are Financial Modeling Prep (FMP) and Finnhub (both often have free or developer tiers).

Financial Modeling Prep (FMP): https://site.financialmodelingprep.com/ + API Documentation: https://site.financialmodelingprep.com/developer/docs
Login (abel) ->

To authorize your requests, add ?apikey=YOUR_API_KEY at the end of every request. API key : G4XwnJmcUpFY8RWvywRZhdnpQ7aVkwNo

Finnhub: https://finnhub.io/ + API Documentation: https://finnhub.io/docs/api/introduction

dubtes que tinc amb Google Sheets

gspread errors

URL's to Google Sheets from python


Google Custom Search API

La Google Custom Search API (també coneguda com a JSON Custom Search API) és la solució oficial de Google per fer cerques de text o d'imatges de forma programàtica. A diferència del scraping (que és el que feies amb DuckDuckGo), aquesta API està dissenyada per a desenvolupadors, és estable i no et bloquejarà per "Ratelimit" si et mantens dins dels marges.

Com funciona?
Per utilitzar-la, no busques directament a "tot Google", sinó que crees un Motor de Cerca Personalitzat (Programmable Search Engine) i després hi connectes el teu script mitjançant una clau de seguretat.

Què necessites per començar?
Per fer-la servir al teu programa Python, necessites dues coses:

Costos
Gratis: Fins a 100 cerques al dia. Per a un script personal o de prova, sol ser suficient.
Pagament: Si necessites més, costa uns $5 per cada 1.000 cerques addicionals.

Configuracio

Per fer servir la Google Custom Search API, has de seguir dos processos: un a la consola de Google Cloud (per la clau) i un altre al configurador de cercadors (per l'ID del motor).

Clau API

  1. Ves a Google Cloud Console {usr Jhngr}
  2. Si no tens cap projecte, crea'n un de nou (clica al desplegable de dalt a l'esquerra -> New Project).
    Posa-li un nom (ex: "projecte per cercar cares").
  3. Al cercador superior, escriu "Custom Search API" i clica-hi.
    Clica el botó Enable (Habilitar).
  4. Un cop habilitada la AOI, ves a la pestanya lateral "Credentials".
    Clica a + Create Credentials -> API Key.
  5. Apareixerà una finestra amb la teva clau (comença per AIza...). Copia-la i guarda-la bé.
Search Engine ID

  1. Ves a Programmable Search Engine
    Clica a Add (Afegir).
    Posa-li un nom (ex: "cercador de cares").
  2. Al camp "What to search?", selecciona "Search the entire web".
  3. Un cop creat el motor (engine), ves a la configuració (Customize) del cercador que acabes de fer.
    Molt important: Busca l'opció Image Search (Cerca d'imatges) i activa-la (ON).
    Busca el camp Search engine ID i copia el codi que apareix.
El neu buscador de cares en python

sebas@minie:~/dades/ai/cares$ ./5_fer.py Llegint fitxer 'noms.txt' Iniciant la cerca per a 2 persones... Buscant cara de: Raphinha Buscant cara de: Vladimir Putin Feina feta: - Creat 'cares.html' amb la taula de resultats. - Creat './urls.txt' amb les URLs de les imatges.

send email - SMTP (and postfix)

Requires a Google low security account

$ cat //MARS//home/nicolau/sebas/python/send_email/alby.py #!/usr/bin/env python3 # # send mail: to whoever, from whoever you think of, and from wherever you want to be :-) # import sys # import socket import smtplib if __name__ == '__main__': print( sys.argv ) if len(sys.argv) != 5: print( "format <from> <to> <subject> <body>\n" ) sys.exit (-1) else: smtpconn = smtplib.SMTP ('localhost') print( "Sending mail from:", sys.argv[1] ) print( "Sending mail to:", sys.argv[2] ) print( "Subject:", sys.argv[3] ) print( "Body:", sys.argv[4] ) print( "in progress..." ) smtpconn.sendmail (sys.argv[1], sys.argv[2], "Subject: " + sys.argv[3] + "\r\n" + sys.argv[4]) print( "done" ) sys.exit (0)

read email using poplib - POP3 protocol client

specs

Minimal example : (from here)

import getpass, poplib M = poplib.POP3('localhost') M.user(getpass.getuser()) M.pass_(getpass.getpass()) numMessages = len(M.list()[1]) for i in range(numMessages): for j in M.retr(i+1)[1]: print(j)

A more detailed code :

sebas@minie:~/dades/python/email/POP_read_mail$ cat gmx_neus.py #!/usr/bin/env python # POP3 client # http://net-informations.com/python/net/pop3.htm # https://www.code-learner.com/python-use-pop3-to-read-email-example/ import poplib import sys from email.parser import Parser pop3server = 'pop.gmx.com' username = 'neus.bertolin@gmx.com' password = 'neus_pwd' pop3server = poplib.POP3_SSL(pop3server) # open connection print( pop3server.getwelcome() ) # show welcome message pop3server.user(username) pop3server.pass_(password) # stat() function return email count and occupied disk size print( 'Messages: %s. Size: %s' % pop3server.stat() ) # list() function return all email list resp, mails, octets = pop3server.list() print( 'email list %s.' % mails ) pop3info = pop3server.stat() # access mailbox status - result is 2 integers: (message count, mailbox size) mailcount = pop3info[0] # get message count print( "Total no. of Email : ", mailcount ) print ("\n=== Start Reading Messages") for i in range(mailcount): print( "\n+++ email index (", i, ")." ) resp, lines, octets = pop3server.retr( i+1 ) # lines stores each line of the original text of the message # so that you can get the original text of the entire message use the join function and lines variable. msg_content = b'\r\n'.join(lines).decode('utf-8') # now parse out the email object. msg = Parser().parsestr(msg_content) # get email from, to, subject attribute value. email_from = msg.get('From') print('From ' + email_from) email_to = msg.get('To') print('To ' + email_to) email_subject = msg.get('Subject') print('Subject ' + email_subject) pop3server.quit() sys.exit()

imap

Mind imap tools

read email using imaplib - IMAP4 protocol client

To access a imap server we can use this code that provides access to imaplib :

\\w500\python\read_emails> type imap.py #!/usr/bin/env python import imaplib import email import sys # set default values my_user = 'myuser@gmail.com' my_pwd = 'mypwdmypwd' my_box = 'inbox' my_server = 'pop.gmail.com' # use command line params if available print( '++++++++++++++++++++ Got ('+ str( len(sys.argv)) +') params.' ) if len(sys.argv) > 1: my_user = str( sys.argv[1] ) if len(sys.argv) > 2: my_pwd = str( sys.argv[2] ) if len(sys.argv) > 3: my_box = str( sys.argv[3] ) if len(sys.argv) > 4: my_server = str( sys.argv[4] ) # display values to use print( '++++++++++++++++++++++++++++++++++++++++ User (' + my_user + ').' ) print( '++++++++++++++++++++++++++++++++++++++++ Pwd (' + my_pwd + ').' ) print( '++++++++++++++++++++++++++++++++++++++++ Box (' + my_box + ').' ) print( '++++++++++++++++++++++++++++++++++++++++ Server (' + my_server + ').' ) print( '(1) *** connect to server (', my_server, ').' ) mmail = imaplib.IMAP4_SSL( my_server ) print( '(2) *** login to user (', my_user, ').' ) mmail.login( my_user, my_pwd ) print( '(3) *** list of folders or gmail labels' ) mmail.list() for i in mmail.list()[1]: l = i.decode().split(' "/" ') print(l[0] + " = " + l[1]) # Hem de anar al sub-folder indicat print( '(4) *** connect to specific folder (', my_box, ').' ) stat, nummsg = mmail.select( my_box ) szNumMsg = str(nummsg) newMsgNum = szNumMsg.split("'")[1] print( '(5) tenim ' + str(newMsgNum) + ' msg' ) result, data = mmail.search( None, "ALL" ) szData = str( data ) newData = szData.split("'")[1] print( '(6) select (', newData, ') msg' ) ids = data[0] # data is a list. # newIds = ids.split("'")[1] print( '(7) IDS (', ids, ').' ) id_list = ids.split() # ids is a space separated string print( '(8) ID LIST (', id_list, ').' ) for num in id_list: print( '(9) Msg ID is (', num, newMsgNum, ').' ) typ, msg_data = mmail.fetch( num, "(RFC822)" ) # fetch the email body (RFC822) for the given ID email_content = msg_data[0][1] for response_part in msg_data: if isinstance( response_part, tuple): msg = email.message_from_string( email_content.decode() ) email_from = msg['from'] email_subj = msg['subject'] print( '(10) subject is (', email_subj, ').' ) print( 'close' ) mmail.close() print( 'logout - shutdown connection to server' ) mmail.logout()

To run this python, we shall use

gmail account has to have less secure apps access enabled !

gmx account has to be accessed with "user@gmx.com" format as userid

delete email using imaplib

Here is the working code :

sebas@minie:~/dades/python/email/IMAP_read_mail/esborrar$ cat delete_files_in_folder.py #!/usr/bin/python3 # https://stackoverflow.com/questions/3180891/imap-how-to-delete-messages from imap_tools import MailBox import imaplib import sys # set default values my_user = 'char@gmail.com' my_pwd = 'wrdfwjygvn' # app pwd ! my_box = 'inbox' my_server = 'pop.gmail.com' # use command line params if available print( '++++++++++++++++++++ Got ('+ str( len(sys.argv)) +') params.' ) if len(sys.argv) > 1: my_user = str( sys.argv[1] ) if len(sys.argv) > 2: my_pwd = str( sys.argv[2] ) if len(sys.argv) > 3: my_box = str( sys.argv[3] ) if len(sys.argv) > 4: my_server = str( sys.argv[4] ) # display values to use print( '++++++++++++++++++++++++++++++++++++++++ User (' + my_user + ').' ) print( '++++++++++++++++++++++++++++++++++++++++ Pwd (' + my_pwd + ').' ) print( '++++++++++++++++++++++++++++++++++++++++ Box (' + my_box + ').' ) print( '++++++++++++++++++++++++++++++++++++++++ Server (' + my_server + ').' ) # from imap_tools import MailBox box = imaplib.IMAP4_SSL( my_server, 993 ) box.login( my_user, my_pwd ) stat, nummsg = box.select( my_box, readonly=False ) total_msg = nummsg[0].decode('utf-8') print(f"() Tenim {total_msg} missatges") szNumMsg = str(nummsg) newMsgNum = szNumMsg.split("'")[1] print( '() tenim ' + str(newMsgNum) + ' msg' ) typ, data = box.search(None, 'ALL') print( '>>> typ is {}'.format(typ) ) print( '>>> data is {}'.format(data) ) for num in data[0].split(): print( '>>> moure a BIN {}/{}'.format(num, newMsgNum) ) box.copy(num, '[Gmail]/Bin') print( '>>> delete {}/{}'.format(num, newMsgNum) ) box.store(num, '+FLAGS', '\\Deleted') box.expunge() box.close() box.logout()
imaplib links

openCV

OpenCV és una llibreria de programació de codi obert dirigida principalment a la visió per computador en temps real, desenvolupada per la divisió russa d'Intel en el centre de Nijni Nóvgorod.

OpenCV homepage, wiki

Intro by Albert {2016.06.02} : OpenCV + "contours" + "webcam"

Per exemple tens el programa "python" corrent, acostes un taulell d'escacs a la webcam, i veus a la pantalla els poligons que trobi el python en temps real. T'estalvies una burrada de hores, a part que es molt divertit i avances molt depressa.

Una mica de codi python :

nicolau@mars:~/sebas/python/openCV$ ls 4 -rwxr--r-- 1 nicolau nicolau 325 Apr 16 10:12 0_versio_opencv.py 4 -rwxr--r-- 1 nicolau nicolau 810 Apr 16 10:13 1_captura_webcam_a_crt.py 4 -rwxr--r-- 1 nicolau nicolau 985 May 24 2020 2_webcam_contours.py 4 -rwxr--r-- 1 nicolau nicolau 53 May 9 2020 3_veure_contingut_cv2.py
OpenCV profiling

OpenCV profiling

Runtime options can be specified via environment variables.

openCV install under Ubuntu

Tutorial

$ sudo apt-get install python-opencv

(20200420) I prefer webcam with Motion + browser

  1. sudo apt update
  2. sudo apt install python-opencv {lets use python 2 by now}
  3. verify the installation :

    nicolau@nicolau-desktop:~/sebas/python/openCV$ python -c "import cv2; print(cv2.__version__)" 3.2.0
openCV demo

Run contours.py

//MARS//home/nicolau/sebas/python/openCV/1_captura_webcam_a_crt.py -> OK

//MARS//home/nicolau/sebas/python/openCV/2_webcam_contours.py -> OK
url

openCV install under Windows

Windows Install - opencv-3.1.0.exe {20160603, win 2k8}

En executar el EXE es crea la carpeta

Compte amb els projectes : The systems PATH contains a list of folders where DLLs can be found.

So we include it in system PATH :

C:\sebas\miscosas\albert\python\webcam> echo %PATH% C:\opencv;C:\python27;C:\ProgramData\Oracle\Java\javapath;c:\eines;C:\Windows\system32; ...

The easiest way to install OpenCV on Windows is to simply copy the cv2.pyd file into python's site-packages directory (e.g. C:\python27\Lib\site-packages). Here is the complete sequence.

Verify openCV install

Code

import cv2 print "### CV2 version I have is " + cv2.__version__

Run it

CV2 version I have is 3.1.0
Display openCV contents

Supose we have this error : AttributeError: 'module' object has no attribute 'namedWindow'

Display CV2 contents using :

  1. enter python command line : python
  2. load cv2 : import cv2
  3. list cv2 contents : help(cv2) -> 2.482 linies !

See //MARS/home/nicolau/sebas/python/openCV/3_veure_contingut_cv2.py

Conclusion : names are Case Sensitive and the exact name is "namedWindow"

openCV functions

opencv documentation !

User Interface te :

contours.py

"contours.py" fa servir aquestes funcions :

cap = cv2.VideoCapture(0) # connecto to webcam cap.isOpened() # ret, frame = cap.read() gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) blur = cv2.GaussianBlur(gray,(5,5),0) ret, thresh_img = cv2.threshold(blur,91,255,cv2.THRESH_BINARY) contours = cv2.findContours(thresh_img,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)[-2] cv2.drawContours(frame, [c], -1, (0,255,0), 3) cv2.imshow('frame title',frame) if cv2.waitKey(1) & 0xFF == ord('q'): cap.release() cv2.destroyAllWindows()
openCV errors

If we get this error :

cv2.error: /build/opencv-VF5Hiu/opencv-2.4.9.1+dfsg/modules/highgui/src/loadsave.cpp:318: error: (-2) could not find a writer for the specified extension in function imwrite_

... it means we specified a destination filename with a wrong extension.

Accepted are ".png", ".jpeg" ...

webCam project

Descripció del nostre projecte :

Android app

I google "android video capture a image every 10 seconds" and find few APPs to do TimeLapse, as

Look for them in Play Store

App - Lapse It

Description:

Features not available in free app:

Storage use calculation :

webcam python code

C:\sebas\miscosas\albert\python\webcam> type veure.py import cv2 print "### CV2 version I have is " + cv2.__version__ import sys print "### SYS version I have is " + str(sys.version_info[0]) + "." + str(sys.version_info[1]) + "." + str(sys.version_info[2]) # 0 = primera webcam cap = cv2.VideoCapture(0) if not cap: print '--- no trobo la webcam' sys.exit(1) while( cap.isOpened() ): # Capture frame-by-frame ret, frame = cap.read() # Our operations on the frame come here imggray = cv2.cvtColor( frame, cv2.COLOR_BGR2GRAY ) # Display the resulting frame if ret: # print dir ( frame ) # display all object fields cv2.imshow( 'my webcam window', imggray ) cv2.waitKey( 50 ) else: print '--- el frame no va be' if cv2.waitKey(50) & 0xFF == ord('q'): print '>>> adeu' break # When everything done, release the capture cap.release() cv2.destroyAllWindows()

FER.CMD has "python veure.py" and says :

import cv2 ImportError: No module named cv2 C:\python27\Lib\site-packages> copy C:\opencv\build\python\2.7\x64\cv2.pyd import cv2 ImportError: DLL load failed: %1 is not a valid Win32 application. C:\python27\Lib\site-packages> copy C:\opencv\build\python\2.7\x86\cv2.pyd import cv2 ImportError: numpy.core.multiarray failed to import C:\python27\Lib\site-packages> python -m pip install --upgrade numpy Collecting numpy Downloading numpy-1.11.0-cp27-none-win32.whl (6.5MB) 100% |################################| 6.5MB 36kB/s Installing collected packages: numpy Successfully installed numpy-1.11.0
webcam input files
webcam project links

motion detection project

Volem generar una alarma en detectar moviment en un video

video recording with motion detection, in Python

RaspberryPi home surveillance with only ~150 lines of Python code - github -> uses Flask - Flask is a micro web framework written in Python

openCV links

I2C and TC74

On our Raspberry, we do :

  1. enable i2c : sudo raspi-config
  2. update Raspbian : sudo apt-get update
  3. install tools : sudo apt-get install i2c-tools
  4. get I2C bus address : i2cdetect -r -y 1
  5. set register 0 to 0x5A : i2cset -y 0x20 0x00 0x5A
  6. sudo apt-get install python-smbus

video 1 , video 2 , python

Some urls : control everything , relay controller , 1 channel , python code

My code at both raspberries


Write into BLOG

Blogger Data API allows client applications to view and update Blogger content - reference

Go to Blogger sag : proves python

W500, 20170126

C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests> run_all_tests.py Traceback (most recent call last): File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests\run_all_tests.py", line 26, in <module> import run_data_tests File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests\run_data_tests.py", line 9, in <module> import gdata_test File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests\gdata_test.py", line 26, in <module> import gdata ImportError: No module named gdata

Improve PYTHONPATH :

C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests> echo %PYTHONPATH% C:\opencv\build\python\2.7\x86\;C:/sebas/miscosas/python/client-code/gdata-python-client-master/src;

Now we have :

C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests> run_all_tests.py Traceback (most recent call last): File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests\run_all_tests.py", line 26, in <module> import run_data_tests File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests\run_data_tests.py", line 19, in <module> import gdata_tests.auth_test File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\tests\gdata_tests\auth_test.py", line 24, in <module> import gdata.auth File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\src\gdata\auth.py", line 29, in <module> import gdata.oauth.rsa as oauth_rsa File "C:\sebas\miscosas\python\client-code\gdata-python-client-master\src\gdata\oauth\rsa.py", line 10, in <module> from tlslite.utils import keyfactory ImportError: No module named tlslite.utils

Get tls lite plus "python setup.py install" : error: Unable to find vcvarsall.bat

Raspberry

We have (20170127)

pi@odin:~/blog $ python -V Python 2.7.9

  1. download the client library
  2. download and install the ElementTree package

Verify the package is already installed with python :

pi@odin:~/blog $ python Python 2.7.9 (default, Mar 8 2015, 00:52:26) [GCC 4.9.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from xml.etree import ElementTree >>>

Download :

pi@odin:~/blog/client $ git clone https://github.com/google/gdata-python-client.git Cloning into 'gdata-python-client'... remote: Counting objects: 10326, done. remote: Total 10326 (delta 0), reused 0 (delta 0), pack-reused 10326 Receiving objects: 100% (10326/10326), 24.60 MiB | 2.27 MiB/s, done. Resolving deltas: 100% (7567/7567), done. Checking connectivity... done. pi@odin:~/blog/client $

Install :

pi@odin:~/blog/client/gdata-python-client $ sudo python ./setup.py install

Run tests :

pi@odin:~/blog/client/gdata-python-client/tests $ ./run_data_tests.py Traceback (most recent call last): File "./run_data_tests.py", line 19, in <module> import gdata_tests.auth_test File "/home/pi/blog/client/gdata-python-client/tests/gdata_tests/auth_test.py", line 24, in <module> import gdata.auth File "/usr/local/lib/python2.7/dist-packages/gdata/auth.py", line 29, in <module> import gdata.oauth.rsa as oauth_rsa File "/usr/local/lib/python2.7/dist-packages/gdata/oauth/rsa.py", line 10, in <module> from tlslite.utils import keyfactory ImportError: No module named tlslite.utils

Get it

pi@odin:~/blog/tls-lite $ wget http://trevp.net/tlslite/tlslite-0.3.8.tar.gz --2017-01-27 10:08:03-- http://trevp.net/tlslite/tlslite-0.3.8.tar.gz Resolving trevp.net (trevp.net)... 204.11.246.1 Connecting to trevp.net (trevp.net)|204.11.246.1|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 637558 (623K) [application/x-gzip] Saving to: âtlslite-0.3.8.tar.gzâ tlslite-0.3.8.tar.gz 100%[=========================>] 622.62K 191KB/s in 3.3s 2017-01-27 10:08:07 (191 KB/s) - âtlslite-0.3.8.tar.gzâ saved [637558/637558]

Uncompress :

pi@odin:~/blog/tls-lite $ tar -xvf tlslite-0.3.8.tar.gz

Install :

pi@odin:~/blog/tls-lite/tlslite-0.3.8 $ ./setup.py install

Still not seen - try again :

pi@odin:~/blog/client/gdata-python-client/tests $ sudo pip install --upgrade tlslite Downloading/unpacking tlslite Downloading tlslite-0.4.9.tar.gz (105kB): 105kB downloaded Running setup.py (path:/tmp/pip-build-89_5kn/tlslite/setup.py) egg_info for package tlslite warning: no files found matching '*' under directory 'docs' Installing collected packages: tlslite Running setup.py install for tlslite changing mode of build/scripts-2.7/tls.py from 644 to 755 changing mode of build/scripts-2.7/tlsdb.py from 644 to 755 warning: no files found matching '*' under directory 'docs' changing mode of /usr/local/bin/tls.py to 755 changing mode of /usr/local/bin/tlsdb.py to 755 Successfully installed tlslite Cleaning up... pi@odin:~/blog/client/gdata-python-client/tests $

Now we have :

pi@odin:~/blog/client/gdata-python-client/tests $ ./run_data_tests.py Traceback (most recent call last): . . . File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1514, in _get with open(path, 'rb') as stream: IOError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/protobuf-3.1.0.post1-py2.7.egg/EGG-INFO/namespace_packages.txt'

So we have permission problems :

pi@odin:~/blog/client/gdata-python-client/tests $ sudo ./run_data_tests.py . . . {very large} test_generate_timestamp (gdata_tests.oauth.data_test.OauthUtilsTest) ... ok ---------------------------------------------------------------------- Ran 39 tests in 0.032s OK
Running the sample code

A full working sample client, containing all the sample code shown in this document, is available in the BloggerExample.py file, under the directory gdata-python-client/samples/blogger/.

You can run the sample with the following arguments:

python BloggerExample.py --email [email_address] --password [password]

El resultat es :

pi@odin:~/blog/client/gdata-python-client/samples/blogger $ python BloggerExample.py --email sebastiasebas@gmail.com --password mypwd Please choose the authorization mechanism you want to use. 1. [deprecated] to use your email address and password (ClientLogin) 2. [deprecated] to use a web browser to visit an auth web page (AuthSub) 3. [deprecated] if you have registed to use OAuth 4. if you have registed to use OAuth 2.0 : 1 Traceback (most recent call last): . . . File "/usr/local/lib/python2.7/dist-packages/gdata/client.py", line 373, in request_client_login_token response, ClientLoginFailed, response_body) gdata.client.ClientLoginFailed: Server responded to ClientLogin request: 404, https://developers.google.com/accounts/docs/AuthForInstalledApps

Googl'ing gets

Important: ClientLogin has been officially deprecated since April 20, 2012 and is now no longer available. Requests to ClientLogin will fail with a HTTP 404 response. We encourage you to migrate to OAuth 2.0 as soon as possible.

Que hem de fer ara ? Anem pel OAuth ! Explicació :

  1. Obtain OAuth 2.0 credentials from the Google API Console {seguir "link" a "explicacio"}
    Visit the Google API Console to obtain OAuth 2.0 credentials such as a client ID and client secret that are known to both Google and your application.
  2. Obtain an access token from the Google Authorization Server.
  3. Send the access token to an API.
  4. Refresh the access token, if necessary.

Tot va be :

4. if you have registed to use OAuth 2.0 : 4 Please enter your OAuth 2.0 Client ID which identifies your app: 51601928208-6qo669nb5u3rv1d7fbotvpt1givuls8u.apps.googleusercontent.c Please enter your OAuth 2.0 Client secret which you share with the OAuth provider: <secret number> + CR Visit the following URL in your browser to authorize this app: https://accounts.google.com/o/oauth2/auth?access_type=offline&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&client_id=5160 What is the verification code? 4/muC2rg5OkbtJmEx5vTIno9GI_DpeLaXvfvt9Ii01BMU ComGiraElMon's Blogs proves de python Successfully created draft post: "Snorkling in Aruba". Successfully created public post: "Back from vacation". Now listing all posts. proves de python Back from vacation No Title primera entrada Now listing all posts between 2007-04-04 and 2007-04-23. 2007-04-04 updated proves de python posts between 2007-04-04 and 2007-04-23 proves de python Now updating the title of the post we just created: Successfully changed the post's title to "The party's over". Now posting a comment on the post titled: "The party's over". Successfully posted "Did you see any sharks?" on the post titled: "The party's over". Now printing all comments Comments on proves de python: The party's over Did you see any sharks? 2017-04-25T03:31:38.593-07:00 Now deleting the comment we just posted Successfully deleted comment. Comments on proves de python: The party's over Now deleting the post titled: "The party's over". Successfully deleted post. proves de python No Title primera entrada pi@odin:~/blog/client/gdata-python-client/samples/blogger $

Request a Google login()

Minie code : /home/sebas/dades/python/google_auth

  1. visit https://console.cloud.google.com/ and log in with your Google account (benet)
  2. click "Select a project" and either choose an existing project or create a new one.
    just request login
  3. go to "APIs & Services" > "Library" (sota les 3 ratlles horitzontals de dalt a l'esquerra)
    search for the Google APIs your app needs (e.g., Gmail API, Drive API) and click "Enable" : Drive and Mail
  4. navigate to "APIs & Services" > "OAuth consent screen"

    • App name := just_login
    • eMail := benet.papell@gmail.com
    • Audience := internal
  5. go to "APIs & Services" > "Credentials" and click "Create Credentials" > "OAuth client ID"
    App type := desktop ; Name := "python_code"
  6. click "Download JSON" : client_secret_530674411431-erbl6f8olnffacldaldco0hodf0m8vck.apps.googleusercontent.com.json

Here is the result :

sebas@minie:~/dades/python/google_auth$ ./3_gemini.py --- Credential Information --- User ID: 107422452309232518508 Email: campdefabes@gmail.com Access Token (DO NOT SHARE): yb29.a0ATi... You are authenticated and ready to make API calls!

Sounds

Where to get them :

omxplayer

Documentation :

sag@odin:~/sounds $ omxplayer -o local buzzer.wav * failed to open vchiq instance sag@odin:~/sounds $ sudo omxplayer -o local buzzer.wav [sudo] password for sag: Audio codec pcm_s16le channels 2 samplerate 44100 bitspersample 16 Subtitle count: 0, state: off, index: 1, delay: 0 have a nice day ;)
aplay
sag@odin:~/sounds $ aplay buzzer.wav xcb_connection_has_error() returned true sag@odin:~/sounds $ sudo aplay buzzer.wav PuTTY X11 proxy: wrong authentication protocol attempted xcb_connection_has_error() returned true Playing WAVE 'buzzer.wav' : Signed 16 bit Little Endian, Rate 44100 Hz, Stereo

Si no funciona, pot ser url :

sag@odin:~ $ sudo amixer controls PuTTY X11 proxy: wrong authentication protocol attempted numid=3,iface=MIXER,name='PCM Playback Route' numid=2,iface=MIXER,name='PCM Playback Switch' numid=1,iface=MIXER,name='PCM Playback Volume' numid=5,iface=PCM,name='IEC958 Playback Con Mask' numid=4,iface=PCM,name='IEC958 Playback Default' sag@odin:~ $

Tambe pot ser

sag@odin:~ $ aplay -l aplay: device_list:268: no soundcards found...
pyglet

pyglet makes it easy to play and mix multiple sounds together in your game. The following example plays an MP3 file :

  1. sudo apt update
  2. sudo apt install pyglet : "E: Unable to locate package pyglet"
  3. sudo pip install pyglet : "Successfully installed pyglet future"
import pyglet music = pyglet.resource.media('music.mp3') music.play() pyglet.app.run()

As with the image loading example presented earlier, pyglet.resource.media locates the sound file in the application's directory (not the working directory). If you know the actual filesystem path (either relative or absolute), use pyglet.media.load :

import pyglet explosion = pyglet.media.load('explosion.wav', streaming=False) explosion.play()

xcb_connection_has_error() returned true

Short sounds, such as a gunfire shot used in a game, should be decoded in memory before they are used, so that they play more immediately and incur less of a CPU performance penalty. Specify "streaming=False" in this case:

import pyglet sound = pyglet.resource.media( 'shot.wav', streaming=False ) sound.play()

url

This example demonstrates playing many short audio samples simultaneously, as in a game.

pygame

import pygame pygame.mixer.init() pygame.mixer.music.load("abc.wav") pygame.music.mixer.play() while pygame.mixer.music.get_busy == True" continue

url with "pygame.mixer.init()" details


python launcher icon

  1. find existing launcher's icon location :

    1. right-click on desktop launcher
    2. select "Open with Text Editor"
      find something like "Icon=cinnamon-panel-launcher"
    3. use "Nemo" to locate that file -> "/usr/share/icons/hicolor/scalable/apps"
  2. use it on a new launcher

Fem servir IA per escriure codi de python

Per llegir PDFs, al Minie fem servir PDFPLUMBER
Per generar ODSs, la llibreria pandas (amb el motor odfpy)

Aqui :

ls /home/sebas/dades/ai/* 4 drwxrwxr-x 2 sebas sebas 4096 Jan 2 19:48 bakup_Rx_a_USB 4 drwxrwxr-x 2 sebas sebas 4096 Jan 13 12:55 bolonyesa 4 drwxrwxr-x 5 sebas sebas 4096 Jan 19 20:34 cares 4 drwxrwxr-x 2 sebas sebas 4096 Nov 6 14:23 diskuse 4 drwxrwxr-x 2 sebas sebas 4096 Jan 8 17:17 factures_bonpreu 4 drwxrwxr-x 3 sebas sebas 4096 Jan 10 17:34 films_quality 4 drwxrwxr-x 2 sebas sebas 4096 Jan 13 20:08 html_taula 4 drwxrwxr-x 2 sebas sebas 4096 Jan 7 12:38 spreadsheet 4 drwxrwxr-x 4 sebas sebas 4096 Feb 3 16:59 thermometre 4 drwxrwxr-x 2 sebas sebas 4096 Nov 6 12:23 ukraine_map 4 drwxrwxr-x 2 sebas sebas 4096 Nov 6 11:23 usa_map 4 drwxrwxr-x 2 sebas sebas 4096 Nov 17 15:13 ww2_map
quina es la millor IA per escriure codi de python

Escollir la "millor" IA per escriure codi Python depèn molt de les teves necessitats específiques, el teu flux de treball i el teu pressupost.
Tanmateix, hi ha diversos líders al mercat que són molt valorats per les seves capacitats:

Opció més popular i versàtil:

Altres opcions destacades amb característiques fortes:

Si la ciència de dades és el teu focus:

una altra opinió

Here are the top 3 AI assistants for Python coding:

Bonus: ChatGPT-4 remains excellent for explaining concepts and debugging assistance.

For pure Python work, I'd recommend starting with Cursor as it's specifically designed for AI-powered coding.

cursor ai

homepage

This site presents Cursor as a cutting-edge AI-powered coding tool designed to make software development more productive by integrating advanced AI models and providing deep codebase understanding.

copilot ai

homepage

an AI companion designed to inform, entertain, inspire, provide advice, feedback, and straightforward answers.

claude ai

homepage

next-generation AI assistant, focused on being safe, accurate, and secure to help users with various tasks including writing, coding, and collaboration.

"prompts" que he fet servir

resultats

sebas@minie:~/dades/ai/thermometre/dibuixar$ ./5_dades_n_fitxers_i_png.py

2024

Tricks


Dubtes python


Links

Ep ! Valid HTML 4.01!   Valid CSS! Escriu-me !
Updated 20201013 (a)  
Uf !