Talk Like A Duck

How to Cure The PayPal Subscription Blues

Posted by Rick DeNatale Mon, 03 Sep 2007 01:08:00 GMT

Since them, I’ve learned more about PayPal subscriptions than I cared to. Today’s revelation on Slashdot of the latest paypal subscription glitch prompts me to share a bit of this hard-earned knowledge.

Todays PayPal Subscription News

It seems that just before the end of August, PayPal deployed some new server code after which ipn (instant payment notification) events for existing PayPal subscriptions stopped being sent to merchants. For many merchants this has caused havoc since the missing payments caused them to cancel subscriptions. One of the things I learned, thankfully before this episode is that if you are using PayPal subscriptions, you don’t want to treat ipn payment notifications as anything but commentary.

How PayPal subscriptions Work

When you offer subscriptions by PayPal you provide a button on your site for users to subscribe. The button performs an HTTP post to the paypal site. The post contains information about the subscription, such as the how much is to be paid, and how often, along with descriptive information about the subscription. PayPal then presents the user with a form page allowing him to accept the subscription, at which point he is returned to your website via a link provided in the post.

In accepting the subscription, the user authorizes PayPal to transfer money to the merchant periodically. When the transfers are attempted, PayPal http posts back to the merchant with instant payment notifications. Everytime a payment succeeds a payment ipn is sent. Payment failures (e.g. the credit card being charged was declined or has expired) cause a payment failure notification, etc. Other notifications are sent when the user cancels a subscription, when the subscription is initially started (subscr_signup), and when it has expired (subscr_eot).

The PayPal protocol isn’t very well specified, at least when it comes to the sequence of notifications for subscriptions. It turns out that it’s not recommended to use payment notifications to track the validity of subscriptions on the merchant’s server. The intent is for PayPal to handle that state, and indicate state transitions with subscr_signup, and subscr_eot. Here are some of the problems which can arise when you try to track subscription state yourself:

1. The order of payment and signup notifications is not specified, I’ve seen the initial payment notification come after the signup notification, but it usually seems to come first. This can cause havoc if you don’t realize this.
2. Let’s say a subscriber signs up for a one-month subscription, then cancels it two weeks later. If you get the cancel notification and turn off the subscription right away, you have taken away two weeks that the subscriber already paid for, and your code might get confused when you get the eot notification at the end of the month. You are supposed to turn the subscription on when you get the signup notification, and turn it off when you get the eot.

   There’s a slight complication to this scheme. If the user signs-up, cancels, and signs up again before the initial sign-up expires, you will still get an eot for the first signup, which means that, if you’re not careful, you can end up turning off the second subscription. My solution to this was to keep a counter of how many unmatched signups the user has, and only turn off the subscription when the counter reached zero.

Fraud Protection

One reason to look at payment notifications is to detect fraud. One of the concerns with paypal is that a user might be able to hijack a subscription signup request and modify the terms. This was one reason that this site was looking at every payment ipn.

Fortunately, the payment term information is also included on signup notifications, so a rigorous check can be made on signup, and a user can be prevented for getting a ten dollar subscription for a one dollar payment. Although the site I’m describing no longer takes automatic action on payment notifications, it still logs them and they are visible via an administrative interface.

Modifying Subscriptions

Another complexity of PayPal subscriptions is how to allow users to modify subscriptions. In fact this is where I came in on this particular web site. They had been offering only monthly subscriptions and wanted to offer an annual subscription with a discount. The key here is to ensure that you tell PayPal that you are modifying a subscription rather than creating a new one. The PayPal documentation describes the ‘modify’ attribute which is to be passed with the subscription signup request post to paypal. A value of ‘1’ for this attribute indicates that it will modify an existing subscription, or create a new one if there is no existing subscription. Don’t beleive this. My initial testing with the PayPal developer sandbox seemed to indicate that this worked. In practice though, once deployed, it always seemed to create a new subscription, which meant that when a user upgraded a monthly subscription to an annual one, she ended up with both.

The fix was to use a value of ‘2’ for modify when the user had an existing subscription, and not to use the modify attribute at all otherwise.

Subscription Modification and Active Merchant

While the ActiveMerchant plugin supports paypal subscriptions, it doesn’t support subscription modification, but adding it was fairly simple. ActiveMerchant uses a mappings hash to map options to the html code it generates for paypal buttons. Here’s a snippet of code from a helper method which adds the modify attribute, and uses it:

    def subscription_form(subscription = Subscribr::Subscription.new, 
              sub_product = Subscribr::SubscriptionProduct.monthly_basic,
              modify=false,
              &proc)
      _erbout = eval('_erbout', proc.binding)
      payment_service_for current_account.id, Subscribr::PayPal.account, :service => :paypal do |service|
        service.mappings[:subscription][:modify] = 'modify'
        service.subscription( {
                                :item => sub_product.name, 
                                :item_name => sub_product.description, 
                                :amount => sub_product.amount_s, 
                                :billing_cycle => sub_product.payment_period, 
                                :billing_cycle_units => sub_product.billing_cycle_units, 
                                :recurring => '1',
                                :reattempt => '1'
                              }.merge(modify ? { :modify => '2' } : {})
                            )
        service.return_url url_for(:only_path => false, 
                   :action => "#{modify ? 'modify' : 'new'}_subscription")
        service.notify_url url_for(:only_path => false, :action => 'ipn')
        service.cancel_return_url url_for(:only_path => false)
        service.return_method '2'
        yield
      end
    end

The Subscrbr module contains application specific code for modeling subscriptions. The helper takes a Subscription object which represents a particular user’s subscription; a SubscriptionProduct which represents a particular type of subscription and its terms. SubscriptionProducts differentiate between monthly and annual subscriptions; and a boolean indicating whether the subscription is to be modified or a new one created.

The line:

service.mappings[:subscription][:modify] = 'modify'

is all that is needed to tell ActiveMerchant how to handle the modify attribute.

Testing PayPal Subscriptions

PayPal is well known for providing a fairly nice testing environment in the form of the PayPal Developers Sandbox. This allows you to set up dummy accounts, and test your site and see what both you, the merchant, and your customer will see.

Unfortunately, this falls down a bit when it comes to recurring payments, since you can’t, as far as I can tell, run the sandbox under an artificial clock and see what happens when the next subscription payment is due, or when the subscription expires.

Is There A Better Way?

Recently Amazon introduced an alternative to PayPal called Amazon Flexible Payment Service which is currently in limited beta testing. Although I haven’t dug into it deeply, it appears to have some interesting differences:

• FPS allows a contract to be set up between a source of funds and a recipient of funds, the contract itself can be initiated(requested) by either the source, the recipient or a third party agent. Shared key cryptography is used to secure establish the contract.
• Once the contract is established transfers are requested by the participants rather than happening automatically. The contract determines whether or not a transfer request is acceptable. For example the contract can specify that a payment of N dollars may be requested by the recipient at most one per month for M months.

It seems to me that the Amazon model is both more flexible, and more open, although as I said, I haven’t investigated it deeply. But given my experiences with PayPal it seems to be worth looking at. One thing that I haven’t worked through with FPS is how a subscription modification would be handled in such a way that the subscriber clearly understands that he is getting a modification rather than an additional subscription.

Summary

But getting back to the main lesson about PayPal subscriptions, don’t use payment notifications to control user access.