This post was written by Hipcamp Staff Product Manager Mike Kolarik, Hipcamp Staff Software Engineer Evgeny Karataev, and Promoted.ai Co-Founder & CEO Andrew Yates.
In 2021, Hipcamp launched personalized search ranking in partnership with Promoted.ai to better match campers with the perfect Hipcamp. Since then, our engineering teams have continued to work together to add new machine learning (ML) features to drive conversion rate improvements.
Most of the time, the impact of these features is unsurprising, if not predictable. For example, it’s no wonder that the price of a Hipcamp influences how likely a camper is to click on a search result. We also weren’t shocked to see that upcoming availability influences how likely a camper is to book. But, every now and then the data surprises us…
Let’s take a look at how we discovered that we could use the sun and the moon to predict conversion.
Our journey starts with a hypothesis: Understanding a location’s weather should allow us to better recommend Hipcamps. If it is extremely hot, maybe campers are more interested in campsites with access to water, for example. So, we use Apple’s WeatherKit API to collect historical and forecast data for each of our Hipcamps using a daily Airflow job. With this, we’re able to collect all of the following weather factors:
Now that we had the raw data, we needed to get it in better shape before integrating it into our ML data pipelines. While sending a 10-day forecast for every Hipcamp is reasonable, it wasn’t reasonable to send daily historical weather data for every Hipcamp. So, we use dbt to transform historical data into weekly aggregates, with the assumption that weekly data would be more meaningful and efficient for our pipeline. Here’s where we landed:
With the data in good shape, it was time to add the new features to our ML pipeline. We decided to create features relative to the current date with the assumption that historical weather data is more useful in this context. Here’s what we added, representative of the code used.
At this point, we made the decision to include some of the more obscure weather features such as sunrise and moonset. After all, it couldn’t hurt—and doing so required no extra effort. Our daily pipeline updates all features and sends them to Promoted.ai via the Item Content Management System (CMS) as JSON in a batch. The JSON documents are converted into numeric, sparse, or categorical features for use in ML models for search ranking. Each day, new models are automatically trained with any new features. If the new model is better at predicting clicks and bookings, it is immediately launched into production.
So, what happened?
To understand the impact of individual weather features in our models, Promoted.ai worked with us to create Feature Importance Reports. The daily reports showed that while straightforward features like “distance” and “historical click-through rate” remain the most impactful, our new features are indeed useful for prediction!
Who would have guessed that the time the moon sets in 5 days could be used to predict how likely someone is to click on a Hipcamp search result?
If you are like us, at this point you may have a healthy sense of skepticism. At Hipcamp, we pride ourselves on being data-driven, but we can’t ignore the impulse to tell a story with the numbers. So, ignoring the perils of interpreting black-box models, we offer the following potentially controversial explanation—going outside is influenced by what’s going on outside.
It’s not a stretch to assume that camping is influenced by the weather. You won’t find a lot of campers in a tent during a snowstorm. Similarly, it’s reasonable to assume that camping is primarily a seasonal activity. We consistently see that things like the appearance of wildflowers in spring and the turning of the leaves in fall impact our business.
So, it shouldn’t be surprising that feeding our models more “outside” features would yield results. After all, there is no single feature that could encapsulate what’s happening outside. Spring in England is different from spring in Arizona, just as June in New South Wales is very different from June in Nova Scotia.
Now that we have the data and we’ve proven its usefulness, it’s time to bring it front and center for Hipcampers in the form of new UX experiences. Our mission to get more people outside depends on our ability to bring the outside into our product. Here are some examples for you:
Stay tuned for an upcoming blog following one product manager on their journey to create a weather widget with the help of a Large Language Model…
The holiday season has arrived in Northern California—the air is crisp, stores are stocking candy canes and eggnog, and the…
Tell Black Friday to take a hike, and get outside this Thanksgiving. Depending on where you're located, camping for Thanksgiving…
Eager to plan your next camping trip but aren't sure where to go? Just look to the stars, which hold…