After completing the lectures on building neural network models using TensorFlow, I was interested in finding out how you communicate the results to a lay audience (i.e. clients.) Specifically, how do you communicate the relative importance of each input (independent variable) in the model? If one variable (e.g., price) has much more of an effect on the output compared to another (e.g., number of books purchased), the client would surely want to know when adjusting their business strategy, etc. How would that difference in impact/effect be quantified and communicated?
Hey Barry,
This is a great question!
In general, there is no direct way to do that. That’s kind of logical as we don’t really know what happens inside the hidden layers (especially when there are thousands of them). That’s the meaning of ‘black box’ and the reason why models like ‘linear regression’ and ‘logistic regression’ are still so popular – they provide lower accuracy but higher interpretation.
However, there are some indirect ways. We want to give you some simple examples, before referring you to further reading.
Example 1:
We trained our model with 10 inputs and got an accuracy of around 80%.
Say, we train it on 9 of the inputs (we omit 1) and get an accuracy of 50% (same as a coin-flip).
It seems, that this 1 input had all the explanatory power.
Example 2:
We trained our model with 10 inputs and got an accuracy of around 80%.
Then we train it on 9 of the inputs (we omit 1, say x1) and get an accuracy of 80%.
What may have happened is that the other variables were retrained in a different way which would yield the same accuracy, but with completely different weights. We can’t say x1 was not important. Rather we can say that x2 to x10 together managed to compensate the loss of predictive power due to the exclusion of x1 for this particular data. In general, x1 may be a useful predictor.
There are many ways to evaluate such relationships, but we don’t really have ideal ones.
Please check this paper for further reading on what may work and what are the potential flaws: ftp://ftp.sas.com/pub/neural/importance.html
Hope this helps!
Best,
The 365 Team
Thank you for your response. The examples you mentioned offer some interesting insights. With respect to the lectures, however, you mentioned that these algorithms can help focus marketing efforts “only on those customers who are likely” to make another purchase, etc. Could you provide some examples of how this would be achieved? How would we identify these customers using information provided by these algorithms?
Hi Barry,
Thanks for the question.
In fact we are just about to publish several more lectures on the topic showing just that! They should be out by next week!
Best,
The 365 Team