About the Job
Job Classification: Year Round
Vail Resorts is looking to hire an Analyst of Channel Testing to lead experimental design and conversion rate optimization (CRO) across marketing channels and enterprise audience segments, enhance predictive modeling and personalization applications, monitor and report on performance of live AB and multivariate experiments, and consult Marketing and Analytics partners on experimental design best practices and process.
- Partner with Analytics and Channel Marketing teams to understand go-to-marketing strategies, business objectives, and key questions. Leverage prior experiment results and new insights to provide new test design recommendations (defining the hypothesis, reporting KPIs, audience segments, traffic splits, etc.).
- Understand Test & Learn tooling and setup process, define success/failure criteria across marketing KPI's, check and mitigate sample bias, advise and measure against confidence levels, and propose audience splits.
- Partner with Analytics, Channel, and IT teams to provide reporting and actionable insights for in-flight experiments. Understand and be able to interpret KPI lifts against confidence levels using statistical methods and business acumen. Advise high-impact audiences for roll-out of winning experiences. Forecast and measure revenue lifts from rolling out winning experiences.
- Partner with Test & Learn teammates to align processes, facilitate trainings, and document best practices that ensure efficient and consistent analysis across the team. Standardize methodology for addressing common questions and ensure that consistent approaches be utilized across the enterprise. Proactively develop new analytic approaches and partner with other analytical groups to develop tools that support marketing strategies at every level.
- Create and distribute recurring experiment read-outs and reporting to track performance and provide insights that inform new learning opportunities for the enterprise.
- Analyze and organize data available for Test & Learn use. Inspect, clean, understand, and transform data as required for analysis purpose. Keep the big picture of business goals in mind while thinking of new ways to approach analysis and enterprise-level insights.
- Support complex ad-hoc pre, during, and post analysis of experiments as required by leadership and partner teams.
- Leverage knowledge of experiment setup, site and campaign implementation, and business KPIs to partner with IT teams to quickly troubleshoot issues as they arise.
- Populate and maintain accessible knowledge base of past experiments, document and forecast revenue impacts against enterprise goals, provide guidance on methodology and assumptions used in analysis, and enforce integrity of Test & Learn process and best practices across the enterprise. Leverage historical learnings to make recommendations on strategy to marketing leadership and lead culture changes to enable an agile Test & Learn environment.
- 1-3 years prior marketing analytics, AB testing, statistics, digital analytics, or other related experience
- Experience evaluating website or marketing channel performance, including analysis of AB experiments and evaluation of results against randomized control groups
- Statistics background including but not limited to AB Testing, Multivariate Testing, Regression Analysis, and Hold-out Experiments required
- Demonstrated success working with very large data sets, developing tools for streamlined analysis, and creating reports that succinctly summarize complex concepts
- Comfortable running basic statistical analysis in Alteryx, RStudio, Python, or other open source analytics software
- Demonstrated strength in written and verbal communication, adjusting message to audience. Excellent interpersonal and relationship management skills, with the ability to communicate effectively at all levels of the organization
- Strong organizational and project management skills are essential. Must be able handle multiple projects simultaneously and coordinate across multiple functions within an organization
- Highly collaborative approach and open communication style
- Comfortable presenting to and communicating with senior leadership
- Hospitality or travel industry experience helpful but not required
- Experience with 1 or more analytics tools/software specific to Test & Learn analysis (Adobe Analytics / Adobe Target / Optimizely / Google Analytics / APT / other CRO platforms) required. RStudio, Python, Alteryx, Tableau, SQL experience a plus.
- High level of comfort with Digital Analytics and Marketing Technology
- High degree of proficiency MS Office Suite, Outlook and Internet applications