Preventing chronic diseases is an essential aspect of medical care. To prevent chronic diseases, physicians focus on monitoring their risk factors and prescribing the necessary medication. The optimal monitoring policy depends on the patient’s risk factors and demographics. Monitoring too frequently may be unnecessary and costly; on the other hand, monitoring the patient infrequently means the patient may forgo needed treatment and experience adverse events related to the disease. We propose a finite horizon and finite-state Markov decision process to define monitoring policies. To build our Markov decision process, we estimate stochastic models based on longitudinal observational data from electronic health records for a large cohort of patients seen in the national U.S. Veterans Affairs health system. We use our model to study policies for whether or when to assess the need for cholesterol-lowering medications. We further use our model to investigate the role of gender and race on optimal monitoring policies.

Continuous tracking of patient’s health data through electronic health records (EHRs) has created an opportunity to predict healthcare policies’ long-term impacts. Despite the advances in EHRs, data may be missing or sparsely collected. In this article, we use EHR data to develop a simulation model to test multiple treatment guidelines for cardiovascular disease (CVD) prevention. We use our model to estimate treatment benefits in terms of CVD risk reduction and treatment harms due to side effects, based on when and how much medication the patients are exposed to over time. Our methodology consists of using the EM algorithm to fit sparse health data and a discrete-time Monte-Carlo simulation model to test guidelines for different patient demographics. Our results suggest that, among published guidelines, those that focus on reducing CVD risk are able to reduce treatment without increasing the risk of severe health outcomes.

Offering promotions has become common practice in the airline industry as a strategy to boost the total revenue. An effective promotion campaign should be adequately priced and timed to attract sufficient extra demand and compensate for the markdown price. Diversion of demand from the regular fare to the markdown price is also a side-effect of offering promotions, which needs to be considered in designing successful campaigns. Demand dilution occurs when customers are attracted to the promotional fare from higher fare families, or from future purchases to the promotional time window. We propose a stochastic dynamic model for the optimal timing of promotions, considering both types of dilution and given fixed prices for the regular and promotional fares. We prove the existence of an optimal policy, and derive structural properties to find the minimum number of unsold seats that justifies the promotion under dilution. We examine the performance of this model on two cases from a Latin American airline and demonstrate considerable savings by applying our proposed optimal policy versus the airline’s current policy.

In the airline industry, deciding the ticket price for each flight directly affects the number of people that in the future will try to buy a ticket. Depending on the willingness-to-pay of the customers the flight might take off with empty seats or seats sold at a lower price. Therefore, based on the behavior of the customers, a price must be fixed for each type of product in each period. We propose a stochastic dynamic pricing model to solve this problem, applying phase type distributions and renewal processes to model the inter-arrival time between two customers that book a ticket and the probability that a customer buys a ticket. We test this model in a real-world case where as a result the revenue is increased on average by 31 percent.