Updating examples for rust-libp2p

For the past week, I was trying to figure out how to update some examples for the Rust implementation of a modular network stack, libp2p.

The Rust implementation is fittingly named rust-libp2p.

What is the issue?

A week ago, an issue in the rust-libp2p repo was opened, requesting some changes for the current examples available in the repository.

These examples were simple applications that demonstrated how one would use rust-libp2p to achieve a certain feature, whether that be a file sharing system, a simple chat system, as well as a very simple ping application, all with the purpose of showcasing the capabilities of the library.

The rust-libp2p library heavily depends on asynchronous libraries to achieve concurrency, so that the client of the library does not have to worry about any threading, while having efficient asynchronous code.

The changes requested in the issue were related to changing the event loop. The responsibility of the event loop is to poll the futures available, such as receiving input from the standard input, or a new peer connecting to the local peer.

If some of the bolded words went over your head, that's totally fine. I will explain right now what they mean:

• poll: in the most general sense, it means asking for something waiting on a queue. For example, if you are doing event-driven programming, there is a part in the computer that takes care of receiving those events to process them. The act of receiving the events in called polling. In our Rust applications, we poll futures.
• future: a future is an abstraction over a computation that may or may not finish. This remark is important. Reading a file is a computation. Most people would call a readFile() function to carry out that computation, which will return you a buffer of bytes, for example. However, what if you wanted to talk about the computation itself? What if you wanted to refer to the readFile() as a concept, and not as an action for the computer to carry out? You may think that we are referring to a closure, but there is a slight difference. A closure refers to the function readFile() as an object that you can pass around, but it doesn't abstract over the fact that it is a computation that may or may not finish. In this case, a closure does not tell you whether a computation can finish, or that it is still being carried out. In Rust, that concept is called a future, defined by the Future trait.

That was a long explanation! If you have some background experience with JavaScript, a Promise is the closest to what the Future trait defines, although there are some differences.

Getting the explanation out of the way, what does this entail for the changes requested? Well, now that you know that for an asynchronous app to work in Rust you need to poll the futures available, this can lead to really boilerplate-like code to arise. If you are using an asynchronous runtime like tokio or async_std, you may use certain functions to handle the scheduling of the polling, but you still need to provide a Future to such functions, which does require some setup when you want to handle several futures at the same time.

One of the files, chat.rs, had an event loop like this:

task::block_on(future::poll_fn(move |cx: &mut Context<'_>| { 
     loop { 
         match stdin.try_poll_next_unpin(cx)? { 
             Poll::Ready(Some(line)) => swarm 
                 .behaviour_mut() 
                 .floodsub 
                 .publish(floodsub_topic.clone(), line.as_bytes()), 
             Poll::Ready(None) => panic!("Stdin closed"), 
             Poll::Pending => break, 
         } 
     } 
     loop { 
         match swarm.poll_next_unpin(cx) { 
             Poll::Ready(Some(SwarmEvent::NewListenAddr { address, .. })) => { 
                 println!("Listening on {:?}", address); 
             } 
             Poll::Ready(Some(SwarmEvent::Behaviour(OutEvent::Floodsub( 
                 FloodsubEvent::Message(message), 
             )))) => { 
                 println!( 
                     "Received: '{:?}' from {:?}", 
                     String::from_utf8_lossy(&message.data), 
                     message.source 
                 ); 
             } 
             Poll::Ready(Some(SwarmEvent::Behaviour(OutEvent::Mdns( 
                 MdnsEvent::Discovered(list), 
             )))) => { 
                 for (peer, _) in list { 
                     swarm 
                         .behaviour_mut() 
                         .floodsub 
                         .add_node_to_partial_view(peer); 
                 } 
             } 
             Poll::Ready(Some(SwarmEvent::Behaviour(OutEvent::Mdns(MdnsEvent::Expired( 
                 list, 
             ))))) => { 
                 for (peer, _) in list { 
                     if !swarm.behaviour_mut().mdns.has_node(&peer) { 
                         swarm 
                             .behaviour_mut() 
                             .floodsub 
                             .remove_node_from_partial_view(&peer); 
                     } 
                 } 
             } 
             Poll::Ready(Some(_)) => {} 
             Poll::Ready(None) => return Poll::Ready(Ok(())), 
             Poll::Pending => break, 
         } 
     } 
     Poll::Pending 
 })) 

If this is your first time reading asynchronous code, I wouldn't blame you if you think that it was difficult to read. The changes requested were directed at this loop: "change the loop to an async-style loop for readability". If you want to know how the end result looks like:

    loop {
        select! {
            line = stdin.next() => match line {
                Some(line) => swarm
                    .behaviour_mut()
                    .floodsub
                    .publish(floodsub_topic.clone(), line?.as_bytes()),
                None => panic!("Stdin closed")
            },
            event = swarm.select_next_some() => match event {
                SwarmEvent::NewListenAddr { address, .. } => {
                    println!("Listening on {:?}", address);
                }
                SwarmEvent::Behaviour(OutEvent::Floodsub(
                    FloodsubEvent::Message(message)
                )) => {
                    println!(
                        "Received: '{:?}' from {:?}",
                        String::from_utf8_lossy(&message.data),
                        message.source
                    );
                }
                SwarmEvent::Behaviour(OutEvent::Mdns(
                    MdnsEvent::Discovered(list)
                )) => {
                    for (peer, _) in list {
                        swarm
                            .behaviour_mut()
                            .floodsub
                            .add_node_to_partial_view(peer);
                    }
                }
                SwarmEvent::Behaviour(OutEvent::Mdns(MdnsEvent::Expired(
                    list
                ))) => {
                    for (peer, _) in list {
                        if !swarm.behaviour_mut().mdns.has_node(&peer) {
                            swarm.remove_node_from_partial_view(&peer);
                        }
                    }
                },
                _ => {}
            }
        }
    }

If you are interested, you can view my PR here.

What I have learned

I have to admit, I actually learned a lot doing this PR! I had to read several posts about how asynchronous programming worked in Rust, since I did not have any prior experience before this. At the start, I couldn't understand the code that much since I wasn't so sure how Rust dealt with asynchronous features and the like. After going through an extensive learning period, I learned just enough to know what changes I should do, which resulted in this PR.