The Social Experiment
The true method of knowledge is experiment – William Blake
While the software is in development we are going to put the methodologies through the paces via a social experiment. This experiment will act as an alpha test, not of the software, but of the different ideologies in use. All of these methods have been proven to work either on their own or in the business world. The data we are gathering will give us a level of quantifiable understanding of how these methods work when used together and/or how they translate from the business world to the personal.
We are under no illusion that the experiments that we conduct will be definitive. As Einstein once famously said, “No amount of experimentation can ever prove me right; a single experiment can prove me wrong.” What we do know is that there is a lot of value in putting these ideas through the rigors of the scientific method.
The Scientific Method
I recently wrote a post on Yuval Harari’s amazing book, Sapiens: A Brief History of Humankind. In the book, he takes us through a wonderful history of the species. One of the epochs that he goes into great detail on was the Scientific Revolution. His statement on this period was brilliant, “The Scientific Revolution has not been a revolution of knowledge. It has been above all a revolution of ignorance. The great discovery that launched the Scientific Revolution was the discovery that humans do not know the answers to the most important questions.”
That’s what we plan to do with this experiment, embrace ignorance. We want to throw out all bias around the methodologies that we know already work and look at this with fresh eyes. Meditation gurus call this ‘beginner’s brain’. When you look at something with beginner’s brain, you are leaving yourself open to new ideas and insights. If this idea sounds new, it shouldn’t. This is basically the definition of the scientific method. We have been using this method for hundreds of years now and it remains the driving force behind discovery.
This is one of my favorite representations of the scientific method
We start at the top with our observations. You can find our observations sprinkled through this site. If I sound a little elusive about the specifics, that’s intentional. We do not want to list the specific questions and hypotheses that we are planning to test so that we don’t bias the tests. Once we make those observations, we start asking the interesting questions. This is a very human faculty, talk to any five year old for a period of time and they will barrage you with a series of whys. Five year-olds are the pinnacle of beginner’s brain, if you ever struggle with trying to find that state, look for your inner five year-old.
Most of us stop after we ask the interesting questions. We either Google ourselves an answer or we chalk it up to the void as an interesting conundrum. The scientific method challenges us to formulate a hypothesis or, with bigger questions, hypotheses. In that process we ask, what do I think is causing this behavior or phenomenon? It does get progressively more difficult from there when we have to develop testable predictions. A testable prediction is very similar to a SMART goal. It needs to be specific, measurable, actionable, relevant and time bound. If it doesn’t fit these requirement we have a snowball’s chance in developing an experiment around it. That’s part of the reason why SMART goals are so effective, because they lend themselves to gathering data. So even if you don’t hit your goal, you are still going to learn something.
Once we have testable predictions we start gathering data. Then, we iterate. In the scientific method, iteration is just a fancy word for failing fast. As Edison said while developing the light bulb, “I have not failed. I’ve just found ten thousand ways that won’t work.” Once we have failed enough that we start to see some interesting trends and patterns that’s when we can develop some general theories.
The Basic Setup of the Experiment
With our alpha social experiment, we have developed our testable predictions and we have already recruited our first victims/test subjects. We will be working with four different groups. Three of the groups represent different demographics – age, gender, relationship status, race, socioeconomic status, etc. The fourth group will be our control group. The control group will answer the same questionnaires as everyone else, so that we can factor in things like seasonality, but they will not be subject to the experiment.
Sample size will certainly be a factor in this first test as the total number of participants will be just shy of 40 people. All the results will have to be taken with that in mind. However, we do plan on extending this test to a far larger public beta. In the second round, even with different predictions, we will be able to counter the issues associated with small sample sizes.
What We Hope to Learn
Obviously, we wish to understand how effective these methods are with certain demographics but the priority of that goal is pretty low on this list. One of the most important things we want to get out of this is understanding which data are the most effective in measuring and predicting success. One of the biggest challenges in building a measure for any goal is finding true lead metrics.
A lead metric is predictive. The partner to a lead metric is a lag metric. A lag metric gives you data after the fact when you can’t do a whole lot about it. A simple example of this is when you are trying to lose weight. Hopping on a scale that tells you your weight is a lag metric. Lead metrics would be things like caloric intake and calories burned through exercise. Another example in business is sales. Lead metrics are leads, there are marketing qualified leads, sales qualified leads, and every other stage of the pipeline but the final stage. The lag metric is total sales, the lead metrics are the earlier stages of the pipeline.
We also wish to test a lot of our predictions on how these methodologies impact other areas of your life. Again, we can’t divulge the specifics of these predictions but we know that there is a ton to learn here.
A final benefit to the experimental approach is that it is a great way to get feedback on the process. We hope that this will teach us what works well, what doesn’t, what is naturally intuitive, what feels forced, etc. This feedback is not indicative of product-market fit but it may be a lead indicator.
Signing Up for the Open Beta Experiment
If you are interested in joining the open beta please hop over to our product page and scroll to the bottom to sign up!