23 KiB
Network and web technologies¶
(I.e. how can we interact with services on other computers from Python?)
Teletype machines "were adapted to provide a user interface to early mainframe computers and minicomputers, sending typed data to the computer and printing the response."
WWII era teletype machines.
Source: https://commons.wikimedia.org/wiki/File:WACsOperateTeletype.jpg
Let's start with an idealized "serial port"¶
Data is carried as a serial stream of bits (1s and 0s, high and low voltage).
We can think about teletype machines, USB, ethernet cables, WiFi/WLAN, and so on:
Source: https://www.expresslrs.org/quick-start/receivers/wiring-up/
(Ethernet cables get more sophisticated. https://serverfault.com/questions/449416/why-do-ethernet-cables-have-8-wires . This is a theme that repeats itself in computer networking.)
But this is not a "network", only two computers
From serial to network¶
We can build a robust network composed of many individual serial links by defining a common language -- namely packets with framing information - such as a destination address - and a payload. The individual nodes in the network use the framing information to forward the packet or to consume it.
This standardization has allowed hardware and software producers to interoperate by having common interfaces. Here is an ethernet frame:
Source: https://commons.wikimedia.org/wiki/File:Ethernet_Type_II_Frame_format.svg
USB also uses the idea of frames to establish a standard for a robust single link "network". Here is a USB frame:
"Data packets would have address field and payload between the packet ID and the end of packet." Source: https://commons.wikimedia.org/wiki/File:USB_signal_example.svg
Source: https://de.m.wikipedia.org/wiki/Datei:Speedport_W_921V,_Deutsche_Telekom-7294.jpg
Source: https://commons.wikimedia.org/wiki/File:Speedport_W_921V,_Deutsche_Telekom-7296.jpg
Source: https://www.conceptdraw.com/How-To-Guide/what-is-a-wireless-network
DHCP - Dynamic Host Configuration Protocol¶
DHCP is the protocol used when your computer first joins a network and asks for an IP address.
Source: https://www.boardinfinity.com/blog/dynamic-host-configuration-protocol-dhcp/
Source: 
These notes give an overview of the most important of these levels with the idea of understanding what your computer is doing when you use the web browser or how a program you write would interact automatically with, for example, a remote server to perform a BLAST query.
TCP and UDP¶
From https://microchipdeveloper.com/tcpip:tcp-vs-udp
In addition to source and destination address, TCP and UDP frames have "port" and thus address has a 2^16 (65536) TCP ports and 2^16 UDP ports.
DNS and IP addresses¶
From https://blog.octo.com/en/comic-introduction-to-networks-ip-addresses/
From https://foxutech.com/what-is-dns-and-how-it-works/
Image source https://twitter.com/b0rk/status/1161997141876903936 (linked from Julia Evans' blog)
-----¶
Image source https://ruslanspivak.com/lsbaws-part1/
-----¶
Image source https://twitter.com/b0rk/status/1155318552129396736
-----¶
Image source https://twitter.com/b0rk/status/1145362860136177664
-----¶
Image source https://twitter.com/b0rk/status/1161679906415218690
-----¶
Image source https://twitter.com/b0rk/status/1145896193077256197
-----¶
Image source https://twitter.com/b0rk/status/1146054159214567424
-----¶
Image source https://twitter.com/b0rk/status/1159839824594915335
-----¶
!curl http://google.com/bananas
Let's practice with the Star Wars API.
curl https://swapi.py4e.com/api/people/1/
We will use the requests library.
import requests
response = requests.get('https://swapi.py4e.com/api/people/5/')
response.text
type(response)
type(response.text)
response.headers
x = response.json()
x
x = response.json()
print(type(x))
x['name']
requests.get('https://swapi.py4e.com/api/people/6/').json()['name']
# webserver1.py from https://ruslanspivak.com/lsbaws-part1/
import socket
HOST, PORT = '0.0.0.0', 4449
listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listen_socket.bind((HOST, PORT))
listen_socket.listen(1)
print(f'Serving HTTP on port {PORT} ...')
while True:
print("waiting for connection")
client_connection, client_address = listen_socket.accept()
print("Got connection from {}".format(client_address))
request_data = client_connection.recv(1024)
print(request_data.decode('utf-8'))
http_response = b"""\
HTTP/1.1 200 OK
Hello, World! Guten tag.
"""
client_connection.sendall(http_response)
client_connection.close()
With telnet:
telnet 127.0.0.1 4448
GET / HTTP/1.1
Host: blahblah.comHTML, CSS, and Javascript and WASM - browser languages¶
HTTP APIs¶
Computer to computer over the internet
Image from https://atesterthing.wordpress.com/2019/03/16/api-and-web-service-definition-difference-and-example/
JSON¶
import json
data1 = {'key1':[1.2,{32: 64,"c":"sdfg'dfsg"},3,4]}
print(type(data1))
buf = json.dumps(data1)
print(type(buf))
# buf
data2 = json.loads(buf)
print(type(data2))
data2
buf
