DEV Community

Cover image for ๏ธ Serverless functions Netlify ๐Ÿš€
Aravind V
Aravind V

Posted on • Updated on • Originally published at devpost.hashnode.dev

๏ธ Serverless functions Netlify ๐Ÿš€

A serverless api to publish my media assets to cloudinary media storage is built part of the this project

Here I am using Netlify functions to demonstrate this capability โœจ

Netlify Status

Why Netlify for this project? ๐ŸŒŸ

Netlify is an all-in-one platform for automating modern web projects. Replace your hosting infrastructure, continuous integration, and deployment pipeline with a single workflow.

Integrate dynamic functionality like serverless functions, user authentication, and form handling as your projects grow.

Millions of developers use Netlify to ship faster, simplify their workflows, and scale effortlessly.

Why publishing to cloudinary? โ˜๏ธ

Beisides acting us a bucket to store media assets on cloud, cloudinary also helps by transform images and videos to load faster with no visual degradation, automatically generate image and video variants and deliver high quality, responsive experience to increase conversions for our websites. Also it also supports AI and ML on the analysis over the media resoure with inhouse and other wellknow solutions in it's market place. Besides that is also offers its web console to manage the media artifacts.

This can also be very much helpful for amateur project developers to have a scalable asset storage, transformation, content delivery solutions on top of their solutions.

You may try the below preview link to understand more on this ๐ŸŽฅ

%[https://cloudinary-res.cloudinary.com/video/upload/f_auto:video,q_auto/v1601900814/website/refresh/dx/Developer_Experience_hero_v2.mp4]

ts-cloud-sync ๐ŸŒฑ

I have only built this as potential capability which i very much require to have scale my amateur blogging solutions without any bottleneck, hopefully in future and yes would make this util open soure once fully ready.

Initially I built this as a express api which uses multer and streamifier to my publish my media to cloudinary. Then i decided to leverage netlify functions to help automate my ci/cd stack and also have a bridge api available always on demand like lamdba and going serverless.

But quickly i realised that deploying to netlify or may be other solutions raise some warning in using multer since it has deprecated buffer feature, which causing a stopper in our working in the last hours of deployment. Hopefully multer will release it v2 soon with stream handling support.

I refractored this into by completely replacing multipart upload feature into streaming upload endpoint, which gave significant advantages in terms of the speed of upload and the memory involved in every invocation.

Because since i pipe the binary streaming data from client end directly into streamifier to furthur pipe to cloudinary v2 upload_stream function.

Eventually i was able to achieve what i have planned for this day long project conceived to participate and share my solution in utilising netlify.

Don't try the endpoint yourself ๐Ÿšง

  • since netlify function endpoint will be validating my authorization key only then cloudinary keys will be supplied to further publish this.

  • The cloudinary asset links are secured and signed based on the access level i may have set for the resource in my presets while i upload.

Don't try the endpoint yourself

Netlify function lambda logs ๐Ÿƒ

Mar 1, 12:06:38 PM: 7d9b959c INFO   /.netlify/functions/index/stream/NetlifyHackathonPoster
Mar 1, 12:06:38 PM: 7d9b959c INFO   Authorization Verified for the resource:  /.netlify/functions/index/stream/NetlifyHackathonPoster
Mar 1, 12:06:38 PM: 7d9b959c INFO   {
  asset_id: '8d7ba9d8201a69febf3f24dd49c29b24',
  mimetype: 'image/jpg',
  asset_size: '124.63 KB',
  public_id: 'private/NetlifyHackathonPoster',
  secure_url: 'https://res.cloudinary.com/dev-post/image/authenticated/s--RDz5Tfma--/v1646116598/private/NetlifyHackathonPoster.jpg',
  Overwritten: false,
  created_at: '2022-03-01T06:36:38Z',
  etag: '844e3ae734b21a11a07192458f1f803e',
  signature: '34139038bbd548fe24ed05fdf8b42d212c9e0426'
}
Mar 1, 12:06:38 PM: 7d9b959c Duration: 471.71 ms    Memory Usage: 73 MB 
Enter fullscreen mode Exit fullscreen mode

When I reupload the same image this will be overwritten so that my resource name is going to be the same.

Mar 1, 12:07:14 PM: 6541e1ec INFO   /.netlify/functions/index/stream/NetlifyHackathonPoster
Mar 1, 12:07:14 PM: 6541e1ec INFO   Authorization Verified for the resource:  /.netlify/functions/index/stream/NetlifyHackathonPoster
Mar 1, 12:07:14 PM: 6541e1ec INFO   {
  asset_id: '8d7ba9d8201a69febf3f24dd49c29b24',
  mimetype: 'image/jpg',
  asset_size: '124.63 KB',
  public_id: 'private/NetlifyHackathonPoster',
  secure_url: 'https://res.cloudinary.com/dev-post/image/authenticated/s--RDz5Tfma--/v1646116598/private/NetlifyHackathonPoster.jpg',
  Overwritten: true,
  created_at: '2022-03-01T06:36:38Z',
  etag: '844e3ae734b21a11a07192458f1f803e',
  signature: '34139038bbd548fe24ed05fdf8b42d212c9e0426'
}
Mar 1, 12:07:14 PM: 6541e1ec Duration: 441.02 ms    Memory Usage: 76 MB 

Enter fullscreen mode Exit fullscreen mode

As a amateur tech blogger i feel this capability to stream uploads to my custom serverless function could be further vertically integrated to automate my publishing process and manage the data at scale and flexibily and securely without relying on other thirdparty hacks.

How I read the file stream inside the function ๐Ÿšก


let file_id: string = publicName(req.params.asset_id);
    async function upload(req: any) {
      let result: any = await streamUpload(file_id, req);
      let logJSON:any = {
        "asset_id": req.params.asset_id,
        "mimetype": `${result.resource_type}/${result.format}`,
        "asset_size": formatBytes(result.bytes),
        "public_id": result.public_id
      }
      if (result.public_id?.length > 0) {
        logJSON = {
          ...logJSON,
          "secure_url": result.secure_url,
          "Overwritten": result.overwritten != undefined ? result.overwritten : false,
          "created_at": result.created_at,
          "etag": result.etag,
          "signature": result.signature,
          "asset_id": result.asset_id,
        }
        console.log(logJSON);
        res.status(200);
        res.json(result);
      } else {
        res.status(400);
        res.json(`{ "error": "Unknown error"}`);
      }
    }

Enter fullscreen mode Exit fullscreen mode

Also find my writeable cloudinary v2 stream created ๐Ÿš 


export const stream_upload = (public_id: string, resolve: any, reject: any) => {
  let stream = cloudinary.v2.uploader.upload_stream(
    {...stream_upload_options,
        public_id: public_id,
    }
    ,
    (error: any, result: any) => {
      if (result) {
        resolve(result);
      } else {
        reject(error);
      }
    }
  );
  return stream;
};


Enter fullscreen mode Exit fullscreen mode

Uploaded screenshots

Status Api with authorization successful

Sample binary stream upload results

Thanks for supporting! ๐Ÿ™

Would be really great if you like to โ˜• Buy Me a Coffee, to help boost my efforts.

๐Ÿ” Original post at ๐Ÿ”— Dev Post

Top comments (0)