Week 8: Finish Direct LiNGAM and Start on Conditional Expectation with ML

 1 Direct LiNGAM

Last week we tried to unveil what behind Direct LiNGAM non-linear algorithm, and figured out what exactly happened for different cases. In this week, I tried to solve the last problem: What if the actual causal relation for x to y is y=f(x+e())? What will happen if we apply our algorithm on this structure? And can we detect this structure?

From the discussion last week, we know that this causal relation can be rewrite as x=f^(-1)(y)-e(). Since the mean of the noise e() is 0, this structure is exactly what we assume correct for causal relation. When I run algorithm on such cases, I would expect that the algorithm will give a reverse direction. However, in experiments, for both directions, the residual and variables are always detected dependently. Since the RCoT algorithm always gives 0 for dependency relationship, we need to find out a new method to evaluate the degree of dependency to determine the direction.

The value Sta we use is a intermediate value inside RCoT function, which is the sum of square of all value in the covariance matrix for Fourier Feature. A larger value would indicate a more strong dependency intuitively.

1.1 Function with no inverse function

It is still possible that the function doesn't have an inverse function. In this case, for both sides of the test, the residual will show strong dependency for both p-value and Sta would be similar and there is no way to determine the direction if only considering the dependency.

1.2 Function with inverse function

When running algorithm on functions with inverse function, like tanh or cubic function, the results turn out to fit our expectation. As we mentioned before that the p-value for both direction would all be close to 0, indicating that for both direction the residual show dependency with variable, which makes it hard to compare. However, when we use Sta to compare the degree of dependency,  we could see that in the backward direction there is less dependence, which fits our expectation that the algorithm will tend to give reverse results.

Since the algorithm for such case just gives out a totally reversed results, there is just no way that I can think of to distinguish between them.

1.3 Merge Request

The pull request for updating RCoT and integrating Direct LiNGAM non-linear has been merged into the Because module main branch.

2 Conditional Expectation with ML

This week I begin to work on this new topic.

By the end of this week, I have implemented the algorithm and run the simple test correctly.

The implementation basically follows other conditional expectation method, while replace the calculation of expectation into the machine learning prediction, and save the model in cache to reuse further.

Some little confusion about implementation arose and can be discussed in the meeting next week.

3 Plan for Next Week

• Discuss and optimize the current implementation
• Design test program to test the performance of conditional expectation.