transaction modifiers miss inferred amounts in transaction

[jkr]

Running with a stack build of up-to-date master.

Transaction modifiers seem to interact oddly with inferred amounts. The following works as expected:

= ^expenses:groceries
    [budget:groceries]                           *-1
    [assets:bank:checking]                        *1

10/7 * MARKET
    expenses:groceries:food                      $20
    assets:bank:checking

$ hledger -ffoo.ledger reg --auto
2018/10/07 MARKET               ex:groceries:food              $20           $20
                                [budget:groceries]            $-20             0
                                [as:bank:checking]             $20           $20
                                assets:bank:checking          $-20             0

But the following does not:

= ^expenses:groceries
    [budget:groceries]                           *-1
    [assets:bank:checking]                        *1

10/7 * MARKET
    expenses:groceries:food
    assets:bank:checking                         $-20

$ hledger -ffoo.ledger reg --auto
hledger: could not balance this transaction - can't have more than one balanced virtual posting with no amount (remember to put 2 or more spaces before amounts)
2018/10/07 * MARKET
    expenses:groceries:food
    [budget:groceries]
    [assets:bank:checking]
    assets:bank:checking                $-20

[simonmichael]

Thanks for the report.

[jkr]

I'll poke around as a way of familiarizing myself with the codebase, and see if I can help out. Would HLedger.Data.TransactionModifer be where I should start? (It looks like the inference logic is in H.D.Transaction, right?)

[simonmichael]

@jkr, that would be great. Yes that sounds right.

[jkr]

I figured out the issue: it was applying the modifiers before it finalized the journal with journalFinalise. A simple solution (apply modifiers after journalFinalise) is here:

[EDIT: this solution doesn't work, for the reasons discussed in this post. There is a better solution in the post below.]

master...jkr:modifiers_fix

Now this works:

= ^expenses:groceries
    [budget:groceries]                           *-1
    [assets:bank:checking]                        *1

10/7 * MARKET
    expenses:groceries:food                      $20
    assets:bank:checking

10/8 * MARKET2
    expenses:groceries:food
    assets:bank:checking                        -$20

$ stack exec hledger -- print -f /tmp/foo.ledger --auto
2018/10/07 * MARKET
    expenses:groceries:food             $20
    [budget:groceries]                 $-20
    [assets:bank:checking]              $40
    assets:bank:checking               $-20

$ stack exec hledger -- print -f /tmp/foo.ledger --auto
2018/10/07 * MARKET
    expenses:groceries:food             $20
    [budget:groceries]                 $-20
    [assets:bank:checking]              $20
    assets:bank:checking

2018/10/08 * MARKET2
    expenses:groceries:food
    [budget:groceries]                 $-20
    [assets:bank:checking]              $20
    assets:bank:checking               $-20

BUT, this produces its own problem. Because the application happens after the finalization, it no longer checks to make sure they balance. So we can now have:

= ^expenses:groceries
    [budget:groceries]                           *-1
    [assets:bank:checking]                        *2

10/7 * MARKET
    expenses:groceries:food                      $20
    assets:bank:checking                        -$20

$ stack exec hledger -- print -f /tmp/foo.ledger --auto
2018/10/07 * MARKET
    expenses:groceries:food             $20
    [budget:groceries]                 $-20
    [assets:bank:checking]              $40
    assets:bank:checking               $-20

So it seems like the best option would be to run finalize twice: before and after modification. This makes the most intuitive sense since the journal should work both with and without --auto, so it's essentially checking two different journals. I'm not sure how expensive this would be, but the second run would only occur if auto_ iopts, so if you don't run specifiy auto you won't see it.

I could also try to split out infernce from finalize, but I imagine there are other checks that would end up missing, so running it twice if you have auto seems like the most direct route.

Thoughts?

Also: when do you use doctests and when do you use easytests? I'll add tests when i make a PR but I'm not sure what you usually do.

[jkr]

Okay, this implements the double-run I discussed above:

master...jkr:modifiers_fix_2

What do you think?

[simonmichael]

That sounds reasonable.

[jkr]

OK -- I put up a PR. There's no testing in at the moment, since I couldn't figure out the proper place to put this test in the current framework. If you have a suggestion, I'll add it to the PR.

[jkr]

I see from the Travis output where the relevant tests are. I'll try to address those and add to the PR.