Project Overview

Recently, I worked on a project to transfer data into different Hubspot test portals with various features such as generating and filtering contacts to add to Portal A, syncing contacts from Portal A to Portal B, and displaying the updated data from each portal. This project used the T3 stack with Next.js, tRPC, and Material UI components.

Issues Discovered

One major issue in my application was the efficiency of updating the contacts to Portal B. The batch API endpoint for contacts is of course the preferred API call to make since it is very efficient in updating large amounts of contacts. However, if there were any mutual contacts shared between the two portals, then the entire batch would fail and no contacts would be synced to Portal B. I needed to create a fallback solution, so my first approach was to use a for..of loop and make a single API request for each contact that we wanted to add from Portal A to Portal B. The issue here is that it takes a very long time for large amounts of contacts, and it often does not feel worth the wait when we have 99 mutual contacts and only 1 new contact--we would have to wait for every single POST request to be completed and then added to the successes or failures arrays in the result object we were returning.

There had to be a better way to handle the fallback and I was made aware of how I could solve this problem in a code review which I was very grateful to hear input from a very experienced engineer who analyzed my code with fresh eyes.

Solution

During the code review, I was asked how could I handle multiple asynchronous requests at once. Initially, I did not understand that he was referring to the POST requests I was making from the for...of loop and instead was thinking about how I had a query and mutation packaged together to pull the data from Portal A and then load it to Portal B. Eventually, I gave an answer about using Promises, to which he then gave me a more detailed answer about using Promise.all(). Still, at that moment I did not fully grasp how that would solve my issue.

After the weekend was over, I decided to brush up on my Javascript and realized that I was not satisfied with my current knowledge of promises, and was very much still bothered that I could come up with a better solution for the application. So, I looked at one of my Javascript courses and focused on the error handling and asynchronous sections. There I started to better understand Promises in general and sharpened my knowledge. Now, I had to figure out how to apply it to my solution and why my reviewer said I should use Promise.all(). Using Promise.all() allows us to run all of the promises concurrently instead of waiting for each one to finish. So, now instead of waiting for every single contact to get handled on the POST request, we can handle them all at the same time. Cool!.. but how? First, let me show you a snippet of my old code that used the for..of loop:

  const successes: Contact[] = [];
  const failures: { contact: Contact; error: string }[] = [];


  const token =
    accountType === "alpha"
      ? process.env.ALPHA_HUBSPOT_API_TOKEN
      : process.env.BETA_HUBSPOT_API_TOKEN;


  for (const contact of contacts) {
    const { firstname, lastname, email, phone } = contact.properties;
    const singleBody = {
      properties: {
        firstname,
        lastname,
        phone,
        email,
      },
    };
    const options = {
      headers: {
        Authorization: `Bearer ${token}`,
        "Content-Type": "application/json",
      },
    };
    try {
      await axios.post(singleContactCreateURL, singleBody, options);
      successes.push(contact);
    } catch (err: any) {
      console.error(
        `Failed to add contact. First Name: ${firstname} \n Last Name: ${lastname} \n Email: ${email}`
      );
      failures.push({
        contact,
        error: err.message || "Failed to add contact",
      });
    }
  }
  return {
    success: failures.length === 0,
    message: `Fallback complete. ${successes.length} succeeded, ${failures.length} failed. `,
    results: {
      successes,
      failures,
    },
  };

The issue with this is there is no way to collect all of the Promises together, we will always just move one after another in this setup. The solution is to build an array of Promises with individual actions for each if it is successful or not. To do this we use the map method to create the array of Promises. Here is the updated code:

  const successes: Contact[] = [];
  const failures: { contact: Contact; error: string }[] = [];


  const token =
    accountType === "alpha"
      ? process.env.ALPHA_HUBSPOT_API_TOKEN
      : process.env.BETA_HUBSPOT_API_TOKEN;
      
  // * create arr of Promises to run all concurrentlyconst contactPromises = contacts.map(async (contact) => {
    const { firstname, lastname, email, phone } = contact.properties;
    const singleBody = {
      properties: {
        firstname,
        lastname,
        phone,
        email,
      },
    };
    const options = {
      headers: {
        Authorization: `Bearer ${token}`,
        "Content-Type": "application/json",
      },
    };
    await axios
      .post(singleContactCreateURL, singleBody, options)
      .then((_) => successes.push(contact))
      .catch((err) => {
        failures.push({
          contact,
          error: err.message || "Failed to add contact",
        });
      });



  });
  await Promise.all(contactPromises);
  return {
    success: failures.length === 0,
    message: `Fallback complete. ${successes.length} succeeded, ${failures.length} failed. `,
    results: {
      successes,
      failures,
    },
  };

Now, I am able to await all the Promises, which in turn drastically shortens the time we must wait to add our contacts. I chose to use the side effects from the POST requests to append to the successes and failures arrays since I wanted to keep the code I already wrote using both arrays. Also, since we are using "async" with our callback, it will ensure that we are returning a Promise despite not having an explicit return value in the map. We build this array of Promises, and then immediately upon creation, we await all the promises. Now, we handle all of these requests concurrently instead of sequentially.

Conclusion

Sometimes, when it comes to solving problems, we get lost in the problem and could really benefit from an outside perspective and allow ourselves time away from the problem to get a fresh approach later. Understanding what your problem is, why it is troublesome, and what an ideal solution would do is essential -- syntax, tips, and specifics come after. Also, acknowledging your weaknesses and proactively working to strengthen those areas are key. I can now say I have a much better understanding of Promises and will continue to push my capabilities and experience of working with Promises in practice and optimizing my network requests.