Daily Surprises and Unexpected Self-Discoveries in Product Management — #2. Economics of a feature

So I worked on a feature to enable certain data (file attachments) added to an onprem client sync over to the cloud so that such attachments could be made available to cloud-based client applications. Essentially, a file sync to the cloud and the ability to access the file via some sort of an API.

The lead engineer designed a fairly robust mechanism and in the end we got the flow to work. The design addressed security concerns as the permissions model between onpremise system and the cloud was different. We solved the fundamental need which was to create a replica of an attachment available onprem in the cloud and vice versa so that same content could be leveraged by cloud and onprem apps.

So, what is the problem? The problem is with the costs of the solution. We used a particular API that does not allow to define the retention policy for the attachments so essentially the attachments are stored infinitely meaning that over time the storage costs keep growing without an easy way to remove the files.

How might I have prevented this? There are a few reasons I failed to realize this problem

1. I have not worked with file management systems in the cloud much in the past so essentially I missed the retention piece of the equation
2. The onprem system where files were initially added to (source system) did not have the retention policy in place so all data was stored infinitely so the engineer working on the feature failed to see this issue as well
3. I did not think of the economics of the feature and its cost factors which is another reason why I failed to see the retention and therefore the particular chosen API as a concern

I did look at the basic flow/happy path for how an attachment syncs over and becomes available to the cloud apps. Ok, so this is the problem. The problem was with the way the problem was framed:

Make files available to cloud apps without a user to have to do double entry (value to the user)

As you can see, there is literally nothing speaking to the value of the business i.e. making the feature economically viable. Have I thought of that, I might have considered the retention aspect as well as provide feedback to the API selected by engineer.

Looking back at the requirement, I am realizing that I also missed the other parts of basic CRUD operations. I focused on add (create) and view (read) without providing clarity around update and delete actions. Luckily, I work with senior engineers who have taken this into consideration and incorporated into their design. As a result, when an attachment is deleted from the source system (onprem), the sync API also removes it from the cloud. As a result, there is some sort of a retention management in place but since there is no onprem enforcement of this behavior, this is too much reliant on the operational practices to occur. And hence prone to NOT happen and hence likely the cloud costs for data storage will keep crawling.

Lessons learnt

1. I need to remember to stay humble. Even after 12 years + of work in requirements and product development, I am still missing requirements
2. I need to remember about CRUD. Any time dealing with any sort of data sync or file and object management, it is important to keep CRUD in mind even if some of it won’t be in scope
3. Use cases and usage scenarios are NOT the only way way to capture requirements. Scenarios are essential and help to capture the story from user’s point of view however there is also a business story to any feature. How does this specific feature help us make money or how does it help to save money and stay afloat.
4. Better communicate with other teams. We were working with someone else’s API. We could have given more context and had somene point out the now obvious (in the hindsight) question. A simple question “what else should we keep in mind working with this API might have triggered more relevant discussions. This is pure speculation though:)

Now I need to talk to the engineer to see how we might handle this moving forward and what possible design changes we should consider.