DEV Community

Steve Layton
Steve Layton

Posted on • Originally published at shindakun.dev on

Polywork Poster in Go

#go

Ploywork Poster in Go

A couple of days ago I finally got access to Polywork. Which, for those who don't know, is like a new-aged Linkedin. It allows you to create a timeline of your work and accomplishments. The idea is that people will be able to contact you through the site to offer you work or opportunities.Take a look at meta.shindakun.dev for a simple example timeline.

One of the things that struck me was that there is no public API at the moment. Which bummed me out as I didn't want to start entering blog posts by hand. So, I decided to poke around a bit and see what the flow is for creating a new post. With the objected being crafting some Go code to create posts.

This would dovetail nicely into our data connector series.

Getting Started

First, we need to look to see how the site lets us put up a new post. This can be done via the network tab in Chromes Dev Tools.

Ploywork Poster in Go

We can see that clicking on the "Post" button brings up a form that we can use to enter our details. Looking at the details of the edit network call shows the entire Turbo Frame that makes up. new in this case seems to just redirect to edit.

Ploywork Poster in Go

Perfect! This will allow us to grab the details we need from the form. We need both the update ID and the authenticity token. Though at present it seems the authenticity token isn't actually required when posting a new update. 🤷‍♂️ I won't show the complete posting flow via the website since it includes cookies but you get the idea.

Ploywork Poster in Go

Our Plan

So here is what we are going to do:

  • make a request to https://www.polywork.com/shindakun/highlights/new/
  • extract the update ID and authenticity token
  • create a multipart form with information we want to post
  • POST to https://www.polywork.com/shindakun/highlights/{{update_id}}
Enjoy this post?
How about buying me a coffee?

Getting Polywork to Go

As always you can find the full code listing at the bottom of this post. Also, this code is pretty sloppy and a bit brittle since it is just a proof of concept.

package main

import (
    "bytes"
    "fmt"
    "io"
    "log"
    "mime/multipart"
    "net/http"
    "regexp"
    "strings"

    "github.com/andybalholm/cascadia"
    "golang.org/x/net/html"
)

const cookie = "_polywork_session=...snip..."
const username = "shindakun"
Enter fullscreen mode Exit fullscreen mode

We start up by setting our cookie value. This is necessary to be able to POST to the website. I simply grabbed this value from the network tab in Chrome. We could start with a "magic" link login but this seemed easier. We'll also set our username.

func main() {
    req, err := http.NewRequest(http.MethodGet, "https://www.polywork.com/"+username+"/highlights/new/", nil)
    req.Header.Add("Cookie", cookie)
    req.Header.Add("Turbo-Frame", "navbar-post-highlight")
    if err != nil {
        log.Fatal(err)
    }

    res, err := http.DefaultClient.Do(req)
    if err != nil {
        log.Fatal(err)
    }
    defer res.Body.Close()
Enter fullscreen mode Exit fullscreen mode

If you've worked with Go before this probably looks very familiar. What we are doing is using http.NewRequest to request to /{{username}}/highlights/new. Note how we are setting the cookie directly in the header and setting a header for the Turbo-Frame.

Calling http.DefaultClient.Do(req) makes the request to the remote server and if successful will return that data in res.

    doc, err := html.Parse(res.Body)
    if err != nil {
        log.Fatal(err)
    }

    // inputs := cascadia.MustCompile("input").MatchAll(doc)

    buttons := cascadia.MustCompile("button").MatchAll(doc)

    re := regexp.MustCompile(`\/shindakun\/highlights\/(.*)\/save_draft`)
    r := re.FindStringSubmatch(buttons[1].Attr[5].Val)
Enter fullscreen mode Exit fullscreen mode

Here we're going to get a bit tricky. The first thing we want to do is use the parse our incoming form as HTML. This then allows us to use the cascadia library and CSS selectors to grab all the buttons. We don't really need all of them but for this first version what the heck.

Using the regular expression \/shindakun\/highlights\/(.*)\/save_draft we extract the ID for the post we are working with.

This is the main section of the code that is very brittle since I'm not doing anything to validate we are looking at the right button key and instead just hard coding the values that I saw during testing.

    // _method: patch
    // authenticity_token: ...snip...
    // draft_highlight[end_date]: 01-01-2021
    // draft_highlight[content]: <div>Testing</div>
    // draft_highlight[activity_ids][]: 76199
    // commit: Post
Enter fullscreen mode Exit fullscreen mode

During testing, I made a copy of the form data POSTed from Chrome for reference. We'll use this as our base form data. As you'll see below not every field is required. I think I can leave out the _method field but have not tried as of yet. I mentioned before but you should also note that I'm not using the authenticity_token token as it seems it is not required. Update: the latest site update now properly requires the token.

    body := &bytes.Buffer{}
    writer := multipart.NewWriter(body)

    fw, _ := writer.CreateFormField("_method")
    _, err = io.Copy(fw, strings.NewReader("patch"))
    if err != nil {
        log.Fatal(err)
    }

    // fw, _ = writer.CreateFormField("authenticity_token")
    // _, err = io.Copy(fw, strings.NewReader(inputs[1].Attr[2].Val))
    // if err != nil {
    // log.Fatal(err)
    // }

    fw, _ = writer.CreateFormField("draft_highlight[content]")
    _, err = io.Copy(fw, strings.NewReader("<div>Posting via Golang</div> :) <a href=\"https://shindakun.dev\">shindakun</a>"))
    if err != nil {
        log.Fatal(err)
    }

    fw, _ = writer.CreateFormField("commit")
    _, err = io.Copy(fw, strings.NewReader("Post"))
    if err != nil {
        log.Fatal(err)
    }

    writer.Close()
Enter fullscreen mode Exit fullscreen mode

It's a bit awkward to create a form in Go but hey, it works. We make a form field with writer.CreateFormField and then io.Copy our data into that field. Right now our payload is hardcoded. We could easily adapt my data connector post to automatically add my Ghost blog posts to Polywork.

    post, err := http.NewRequest("POST", "https://www.polywork.com/shindakun/highlights/"+r[1], bytes.NewReader(body.Bytes()))
    if err != nil {
        log.Fatal(err)
    }
    post.Header.Set("Content-Type", writer.FormDataContentType())
    post.Header.Add("Cookie", cookie)
    rsp, _ := http.DefaultClient.Do(post)
    if rsp.StatusCode != http.StatusOK {
        log.Printf("Request failed with response code: %d", rsp.StatusCode)
    }
}
Enter fullscreen mode Exit fullscreen mode

And now we do the actual form submit. And that's all there is to it.

Ploywork Poster in Go

What's Next?

Next, I think I am going to adapt this code to use with the data connector we are working on. I'll probably try to make it pretty generic so I can use it as its own module.

Code Listing 

package main

import (
    "bytes"
    "fmt"
    "io"
    "log"
    "mime/multipart"
    "net/http"
    "regexp"
    "strings"

    "github.com/andybalholm/cascadia"
    "golang.org/x/net/html"
)

const cookie = "_polywork_session=...snip..."
const username = "shindakun"

func main() {
    req, err := http.NewRequest(http.MethodGet, "https://www.polywork.com/"+username+"/highlights/new/", nil)
    req.Header.Add("Cookie", cookie)
    req.Header.Add("Turbo-Frame", "navbar-post-highlight")
    if err != nil {
        log.Fatal(err)
    }

    res, err := http.DefaultClient.Do(req)
    if err != nil {
        log.Fatal(err)
    }
    defer res.Body.Close()

    doc, err := html.Parse(res.Body)
    if err != nil {
        log.Fatal(err)
    }

    // inputs := cascadia.MustCompile("input").MatchAll(doc)

    buttons := cascadia.MustCompile("button").MatchAll(doc)

    re := regexp.MustCompile(`\/shindakun\/highlights\/(.*)\/save_draft`)
    r := re.FindStringSubmatch(buttons[1].Attr[5].Val)

    // _method: patch
    // authenticity_token: ...snip...
    // draft_highlight[end_date]: 01-01-2021
    // draft_highlight[content]: <div>Testing</div>
    // draft_highlight[activity_ids][]: 76199
    // commit: Post

    body := &bytes.Buffer{}
    writer := multipart.NewWriter(body)
    fw, _ := writer.CreateFormField("_method")
    _, err = io.Copy(fw, strings.NewReader("patch"))
    if err != nil {
        log.Fatal(err)
    }
    // fw, _ = writer.CreateFormField("authenticity_token")
    // _, err = io.Copy(fw, strings.NewReader(inputs[1].Attr[2].Val))
    // if err != nil {
    // log.Fatal(err)
    // }
    fw, _ = writer.CreateFormField("draft_highlight[content]")
    _, err = io.Copy(fw, strings.NewReader("<div>Posting via Golang</div> :) <a href=\"https://shindakun.dev\">shindakun</a>"))
    if err != nil {
        log.Fatal(err)
    }
    fw, _ = writer.CreateFormField("commit")
    _, err = io.Copy(fw, strings.NewReader("Post"))
    if err != nil {
        log.Fatal(err)
    }
    writer.Close()

    post, err := http.NewRequest("POST", "https://www.polywork.com/shindakun/highlights/"+r[1], bytes.NewReader(body.Bytes()))
    if err != nil {
        log.Fatal(err)
    }
    post.Header.Set("Content-Type", writer.FormDataContentType())
    post.Header.Add("Cookie", cookie)
    rsp, _ := http.DefaultClient.Do(post)
    if rsp.StatusCode != http.StatusOK {
        log.Printf("Request failed with response code: %d", rsp.StatusCode)
    }
}
Enter fullscreen mode Exit fullscreen mode

Top comments (1)

Collapse
 
altiano profile image
Altiano Gerung •

interesting,

wondering if you also will automate the "getting the cookie" process?

Read next

neelp03 profile image

ThrottleX: Scaling to a Million Requests Per Second Without Breaking a Sweat

Neel Patel -

tuna99 profile image

Understanding Iterators in Go: A Fun Dive!

Anh Tu Nguyen -

mohamedfawas1 profile image

Building a Robust E-Commerce Platform with Go, Clean Architecture, and PostgreSQL

MOHAMED FAWAS -

huizhou92 profile image

Decrypt Go: varint

huizhou92 -