Active Sync Upgrade - Making it All Possible
Rapid transfer of product data is key in ecommerce and the reason why we invested heavily in rebuilding our infrastructure. The new generation launched is called Active Sync.
Possibly the fastest and smoothest sync of its kind, Active Sync is essentially a big data ETL pipeline built on top of the Apache Pulsar streaming platform. It makes it possible to update changes for individual products, without having to reindex the full database.
Active Sync has a positive effect on speeding up bulk updates (from a bulk API, CSV, JSON etc.) since updates begin as soon as individual product data has been downloaded, without need to wait for the full batch to be available. This is of course valuable but the real game changer comes when paired with real time data updates (e.g. webhooks or inserting data through the Push API).
Norce Webhooks Now Available
To enable these benefits out of the box for all merchants, Norce has built out native webhooks.
This will further improve the Findify <> Norce integration and is available from March 2023.
The first generation webhooks includes 4 different values:
- Price: Original / Full Price
- Sale_Price: Discounted Price
- Quantity: Number of Available Products
- Availability: In stock / Out of Stock
How Does It Work?
Norce Webhooks Require SKU = ID
The Norce webhooks are only sending SKU as product identifier. This cannot be configured. In order to connect the price/availability update to the correct variant, this require a Findify <> Norce client to use the SKU field as the Findify ID field!
Read more about standard fields
When one of these fields are updated in Norce (directly through the Dashboard or via an integrated PIM / ERP), a webhook is fired into Findify. The webhook contain the Store ID, Channel ID, SKU and the updated product data.
This product data is pushed directly into Findify's index and will be available in seconds to a few minutes.
Q: What happens if I update a lot of products at the same time? Will that break the Sync?
A: No, that will not break the sync. Product data entering the system through webhooks goes through the same flow as bulk updates, which is built for large volumes. If multiple products are being sent through, we will process them in batches of either 1,000 product updates or 1min, whatever comes first.
Q: Is there a risk that a delay in my regular product sync overrides webhook updates, changing it back to old data?
A: No, this has been encountered for in the design. During daytime, data updates of Price/Sale_price & Quantity/Availability is fully disregarded. The exception is for newly added products for which all data is synced.
Q: Does campaign Prices in Norce fire Webhooks?
A: No. Norce process campaign prices & discounted prices in the pricelist differently. CAMPAIGN PRICES DOES NOT FIRE WEBHOOKS. Campaigns are applied to the products in the frontend and as such can’t currently push a webhook on it. Since most campaign prices are activated during nighttime, we’ve created a special nighttime window for full product updates, i.e. also including the price information from Hyperdrive.
Q: How should I work with campaign prices given that they aren't firing webhooks?
A: Campaign prices are treated differently in Norce and does not fire webhooks. Campaign prices should be activated/deactivated during the night time window, preferably already at midnight to properly sync the correct price data.
Q: What is daytime vs. nighttime sync?
A: The sync is designed to treat price changes from the feed differently depending on the time of day. During daytime they are disregarded to not "sync back" to an old price. During nighttime they are included to ensure campaign prices kicks in.
Q: Can the night time window be configured?
A: Yes. By default, the nighttime window is set from midnight to 06:00. However, this can easily be adjusted per merchant through a simple update from the support team.
Activating Webhooks is fairly smooth!
Key areas are ensuring you're using the values Norce sends in the webhook (SKU) in your Findify setup as the unique ID field. And starting with testing in your staging account to validate all configurations are in tune.
Steps to activate Webhooks for Norce:
- Ensure you're using the Norce SKU field as your Findify ID field. Read more about standard fields
- If changed, ensure to update the same in the Analytics Tracking
- Findify will need to update your account to the latest sync version (built for the daytime vs. nighttime windows). This is quick but should always start with validation in the staging account.
- Norce activates webhooks, which is done per merchant.
- A few products are updated in the account to validate all configurations are correct.
Q: Can Findify process more than Price/Sale_price & Quantity/Availability in direct updates
A: Yes. Findify can process all fields (incl. custom) in direct updates of individual products through webhooks or our Product Push API ]
Updated 6 days ago