Channels ▼
RSS

Design

The New asyncio in Python 3.4: Servers, Protocols, and Transports


Whenever a client connects, the client_connected_handler will run with the following two arguments, which provide a (reader, writer) pair and allow you to use the stream's high-level API:

  • client_reader: An asyncio.StreamReader object.
  • client_writer: An asyncio.StreamWriter object.

The code in the client_connected_handler coroutine displays a message indicating that a client connection has been received and enters a loop that calls another coroutine, the client_reader.read method. It uses the yield from keywords to wait for the StreamReader object to read up to 8192 bytes without blocking. When the client_read.read coroutine finishes its execution, the code displays the read data and makes a call to client_writer.write to send the previously read data back to the client. The client_writer.write method writes the data bytes received as an argument to the transport without blocking. The method calls the corresponding transport method that buffers the data and arranges for it to be sent out asynchronously. If there is no more data to read, the while loop breaks its execution. This way, the server sends the data received back to the client .

Now, let's move on to the client code. After creating an event loop named loop, the code makes a call to loop.run_until_complete to execute the simple_echo_client coroutine. This coroutine calls the asyncio.open_connection coroutine, which creates a streaming transport connection to the host and port specified as arguments with the socket type set to SOCK_STREAM. A successful connection returns a (reader, writer) pair. The code uses the yield from keywords to wait without blocking for the asyncio.open_connection coroutine to create the streaming transport connection with the StreamReader and StreamWriter objects associated with the transport and protocol. Then, the code calls writer.write many times to send a few lines of text to the transport.

Because the client expects the server to send back some data, the code enters a loop that calls another coroutine, the reader.readline method. Again, the code uses the yield from keywords to wait for the StreamReader object to read a sequence of bytes ending with \n or until EOF is received. The code displays each read line and the while loop breaks when this line is equal to the one defined as the last line. In order to make the code easier to read, I haven't included all the necessary try…finally blocks. As I explained in the previous installment, you can ignore the existence of the yield from keywords for exception handling purposes. If an exception occurs within any of the coroutines called with the yield from keywords, it will be raised the same way as when you don't use yield from.

Working with asyncio.Task

As I explained in the previous installment, an asyncio.Task is a coroutine wrapped inside a future and runs as long as the event loop runs. The following lines show an example of another TCP echo server with a line oriented protocol that uses tasks, coroutines, and callbacks. You can execute the following lines in a Python console and then run a telnet client to localhost on port 2222 as explained in the first example. Enter some text, press Enter, and you will see the line coming back from the server. In this case, the code doesn't display messages in the console and simply focuses on handling the connection and the stream readers and writers.

import asyncio

clients = {} # task -> (reader, writer)

def client_connected_handler(client_reader, client_writer):
    # Start a new asyncio.Task to handle this specific client connection
    task = asyncio.Task(handle_client(client_reader, client_writer))
    clients[task] = (client_reader, client_writer)

    def client_done(task):
        # When the tasks that handles the specific client connection is done
        del clients[task]

    # Add the client_done callback to be run when the future becomes done
    task.add_done_callback(client_done)

@asyncio.coroutine
def handle_client(client_reader, client_writer):
    # Handle the requests for a specific client with a line oriented protocol
    while True:
        # Read a line
        data = (yield from client_reader.readline())
        # Send it back to the client
        client_writer.write(data)

loop = asyncio.get_event_loop()
server = loop.run_until_complete(asyncio.start_server(client_connected_handler, 'localhost', 2222))
try:
    loop.run_forever()
finally:
    loop.close()

clients keeps track of all the clients that are connected to the server with tasks of (reader, writer) pairs. It is usually useful for performing specific operations with clients, such as killing their connections or broadcasting data to all of them. After creating an event loop named loop, the code makes a call to loop.run_until_complete to call the asyncio.start_server coroutine, which starts a socket server bound to the specified host and port and executes the callback specified as an argument for each client connected, client_connected_handler. As happened in the previous example, client_connected_handler is another couroutine and it will be automatically converted to a Task.

By this point, you already know that the client_connected_handler receives the two arguments that provide a (reader, writer) pair. In this case, the code starts a new asyncio.Task to handle this specific client connection; that is, a task of a (reader, writer) pair. The task executes the handle_client coroutine with client_reader and client_writer as the arguments. The code saves the task of a (reader, writer) pair in clients, and calls the task.add_done_callback method to add the client_done callback to be run when the future is done. Thus, when handle_client finishes its execution, the client_done callback will be executed and will remove the task from clients. In more complex scenarios, you can use the saved tasks of (reader, writer) pairs to perform different kinds of operations with all the connected clients. The handle_client code is easy to understand because it just uses the StreamReader and StreamWriter objects to read a line from the client and write back to it with a line-oriented protocol.

Conclusion

The new asyncio module reboots coding asynchronous I/O with Python and provides an easy-to-use high-level streams API. In truth, it provides everything you need to work with the most common I/O scenarios without installing additional packages. However, when you need greater control for complex scenarios, you can take advantage event loop policies, tasks, specialized synchronization primitives, and specific logging and debugging features. If you want to dive deeper on the additional features provided by the module, browse the many advanced examples in the Google Code repository for asyncio. The repository still uses the previous name for the module (Tulip), but the examples still work with the version included in Python 3.4.


Gastón Hillar is a senior contributing editor at Dr. Dobb's.

Related Article

The New asyncio Module in Python 3.4: Event Loops


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video