Skip to content

Conversation

@0xlwu
Copy link
Contributor

@0xlwu 0xlwu commented Jul 23, 2021

This is a potential implementation of proposals (6) and (10) from #1559. These commits handle the balance family of commands but should be extendable to compound-balance family as well.

@simonmichael simonmichael added A-WISH Some kind of improvement request or proposal. balance labels Jul 24, 2021
@simonmichael
Copy link
Owner

This looks like an excellent start!

I tried to understand the new code just by reading the names and haddocks, and had trouble. Perhaps we can find ways to make some of those clearer.

The terminal output shows (fake) subaccounts, while the CSV output shows a separate commodity column. Should we have them both do the same thing ? Which is better ? Will the fake accounts be confusing/frustrating, eg will people try to filter by those accounts or view their register ? If the subaccounts are a good idea, would we want to implement them more deeply so they are visible in all reports ?

@Xitian9
Copy link
Collaborator

Xitian9 commented Jul 24, 2021

I think a good approach would be to extend showMixedAmountB and showMixedAmountLinesB so that you can specify the order they display amounts in. That is a flexible approach which could be easily adapted to any of the proposed multi-line display methods.

I'm personally in favour of a commodities column instead of fake subaccounts, but if others prefer subaccounts I wouldn't object.

@simonmichael
Copy link
Owner

A column seems simpler and a more limited scope than an account name. However it doesn't work for tree-mode balance reports, am I right ?

@Xitian9
Copy link
Collaborator

Xitian9 commented Jul 25, 2021

I think we have big problems for tree-mode balance reports with any method. For example, if we have 1 USD, 1 CAD in an account assets:bank, how do we display the inclusive balances in assets? Splitting into accounts as in this PR will not work. A column would work even in tree mode, though it would make for very long reports.

@simonmichael
Copy link
Owner

We have discussed three places to show commodity symbols.

  1. Inline in each amount. Current behaviour.
  2. In a separate column. Implemented by this PR for CSV output. Works only with reports in list mode.
  3. As a fake (generated) subaccount. Implemented by this PR for terminal output. Might or might not need deeper support of these fake subaccounts for other commands to avoid confusion.

If we will ultimately want all three of these, we should provide a three-way option, like --show-commodity-in=amount|column|account.

It seems to me the simplest lowest-impact way to address #1559 is to implement the column mode for both terminal and CSV output. It should give an error when used with unsupported output formats (json...) or incompatible modes (tree). It would be ok to start with this and change our minds later I guess.

Thoughts ?

@0xlwu
Copy link
Contributor Author

0xlwu commented Jul 26, 2021

I agree that tree mode would make this 'fake subaccount' view quite confusing. If we adopted the syntax of (9), i.e commodity in brackets after, that could resolve that issue.

However, I think that having a consistent column for csv and terminal output is probably the least surprising behavior. I'll rework this to take some of Xitian9's suggestions and the commodity column in both.

@simonmichael
Copy link
Owner

simonmichael commented Jul 26, 2021 via email

@Xitian9
Copy link
Collaborator

Xitian9 commented Jul 27, 2021

@simonmichael: I think we agree on the course of action to take, but I'm actually not able to picture how subaccounts would work in tree mode. Could you do a quick mockup of how this should appear in tree mode? Assume that we're calling it with --no-elide so that there is no boring parent elision.

$ hledger bal -T -Y --commodity-account
Balance changes in 2012-01-01..2014-12-31:

                       ||     2012    2013     2014    Total
=======================++====================================
 Assets:US:ETrade:GLD  ||        0   70.00        0    70.00
 Assets:US:ETrade:USD  ||   337.18  -98.12  4881.44  5120.50
-----------------------++------------------------------------
 total:GLD             ||        0   70.00        0    70.00
 total:USD             ||   337.18  -98.12  4881.44  5120.50

@0xlwu
Copy link
Contributor Author

0xlwu commented Jul 27, 2021

In the case of tree mode, should the account name be repeated? e.g

                   || Commodity              2012              2013              2014
===================++=================================================================
 Assets:US         || GLD                       0             70.00                 0
 Assets:US         || ITOT                  10.00             18.00            -11.00
 Assets:US         || RGAGX      139.973000000000  171.689000000000  178.295000000000
 Assets:US         || USD                 7785.76           -299.58          -1769.65
 Assets:US         || VACHR                120.12            120.12             97.02
 Assets:US         || VBMPX      111.933000000000  101.000000000000   97.017000000000
 Assets:US         || VEA                   12.00             10.00             14.00
 Assets:US         || VHT                  106.00             18.00            170.00
   BofA:Checking   || USD                 7448.62           -201.50          -6651.07
   ETrade          || GLD                       0             70.00                 0
   ETrade          || ITOT                  10.00             18.00            -11.00
   ETrade          || USD                  337.18            -98.12           4881.44
   ETrade          || VEA                   12.00             10.00             14.00
   ETrade          || VHT                  106.00             18.00            170.00
     Cash          || USD                  337.18            -98.12           4881.44
     GLD           || GLD                       0             70.00                 0
     ITOT          || ITOT                  10.00             18.00            -11.00
     VEA           || VEA                   12.00             10.00             14.00
     VHT           || VHT                  106.00             18.00            170.00
   Hoogle:Vacation || VACHR                120.12            120.12             97.02
   Vanguard        || RGAGX      139.973000000000  171.689000000000  178.295000000000
   Vanguard        || USD                   -0.04              0.04             -0.02
   Vanguard        || VBMPX      111.933000000000  101.000000000000   97.017000000000
     Cash          || USD                   -0.04              0.04             -0.02
     RGAGX         || RGAGX      139.973000000000  171.689000000000  178.295000000000
     VBMPX         || VBMPX      111.933000000000  101.000000000000   97.017000000000

vs

                   || Commodity              2012              2013              2014
===================++=================================================================
 Assets:US         || GLD                       0             70.00                 0
                   || ITOT                  10.00             18.00            -11.00
                   || RGAGX      139.973000000000  171.689000000000  178.295000000000
                   || USD                 7785.76           -299.58          -1769.65
                   || VACHR                120.12            120.12             97.02
                   || VBMPX      111.933000000000  101.000000000000   97.017000000000
                   || VEA                   12.00             10.00             14.00
                   || VHT                  106.00             18.00            170.00
   BofA:Checking   || USD                 7448.62           -201.50          -6651.07
   ETrade          || GLD                       0             70.00                 0
                   || ITOT                  10.00             18.00            -11.00
                   || USD                  337.18            -98.12           4881.44
                   || VEA                   12.00             10.00             14.00
                   || VHT                  106.00             18.00            170.00
     Cash          || USD                  337.18            -98.12           4881.44
     GLD           || GLD                       0             70.00                 0
     ITOT          || ITOT                  10.00             18.00            -11.00
     VEA           || VEA                   12.00             10.00             14.00
     VHT           || VHT                  106.00             18.00            170.00
   Hoogle:Vacation || VACHR                120.12            120.12             97.02
   Vanguard        || RGAGX      139.973000000000  171.689000000000  178.295000000000
                   || USD                   -0.04              0.04             -0.02
                   || VBMPX      111.933000000000  101.000000000000   97.017000000000
     Cash          || USD                   -0.04              0.04             -0.02
     RGAGX         || RGAGX      139.973000000000  171.689000000000  178.295000000000
     VBMPX         || VBMPX      111.933000000000  101.000000000000   97.017000000000

Also, in the normal balance mode with text rendering, do we agree that the current output is OK or should the commodity be aligned separately? e.g

         5120.50 USD  Assets:US:ETrade:Cash
           70.00 GLD  Assets:US:ETrade:GLD
          17.00 ITOT  Assets:US:ETrade:ITOT
           36.00 VEA  Assets:US:ETrade:VEA
          294.00 VHT  Assets:US:ETrade:VHT

vs

         5120.50  USD  Assets:US:ETrade:Cash
           70.00  GLD  Assets:US:ETrade:GLD
           17.00  ITOT Assets:US:ETrade:ITOT
           36.00  VEA  Assets:US:ETrade:VEA
          294.00  VHT  Assets:US:ETrade:VHT

@Xitian9
Copy link
Collaborator

Xitian9 commented Jul 27, 2021

For the multi-column report, I think the second is more consistent with how the normal balance report handles things, and saves a lot of clutter. For the question of aligning columns I don't have a strong opinion, but maybe we should leave this question aside as a minor stylistic tweak until we've sorted the primary behaviour.

I think your tree example is an excellent demonstration of the fact that we probably shouldn't worry too much about how it looks in tree mode. It will be clunky and verbose in any case, and flat mode is the primary use case for commodity columns.

@simonmichael
Copy link
Owner

simonmichael commented Jul 27, 2021 via email

@Xitian9
Copy link
Collaborator

Xitian9 commented Jul 27, 2021

I've run the branch locally for a few sample reports, and I have to say it's looking great so far! Very clean and concise. I'll give more in-depth feedback when I've had a chance to look in the code in detail, but the results are great.

@Xitian9
Copy link
Collaborator

Xitian9 commented Jul 27, 2021

Is it convenient to squash some of your commits so that changes which are later overwritten disappear? That would make review a bit easier.

@Xitian9
Copy link
Collaborator

Xitian9 commented Jul 27, 2021

We seem to have lost amount elision in ordinary single-line multi-column balance reports. In particular test/balance/multicommodity.test is now failing in tests 1 and 3.

@0xlwu 0xlwu force-pushed the commodity-account branch from 570ca1e to f56c28a Compare July 27, 2021 15:47
@0xlwu
Copy link
Contributor Author

0xlwu commented Jul 27, 2021

Thanks for taking a look. I've fixed the regression and consolidated the git history as well.

Do we care to label everything as commodity-column now?

@simonmichael
Copy link
Owner

simonmichael commented Jul 27, 2021 via email

@0xlwu 0xlwu force-pushed the commodity-account branch from f56c28a to 1e93581 Compare July 27, 2021 18:16
@simonmichael
Copy link
Owner

simonmichael commented Jul 28, 2021

Quick test notes, if not out of date:

  • --commodity-column help needs updating
  • currently affects only csv output
  • adds the commodity column, but does not yet remove the commodity symbol from the balance column

@0xlwu
Copy link
Contributor Author

0xlwu commented Jul 28, 2021

  • currently affects only csv output

What do you mean by this?

  • --commodity-column help needs updating
  • adds the commodity column, but does not yet remove the commodity symbol from the balance column

Done, thanks!

@simonmichael
Copy link
Owner

simonmichael commented Jul 28, 2021 via email

@Xitian9
Copy link
Collaborator

Xitian9 commented Jul 28, 2021

It was definitely affecting terminal output on the version I saw.

@simonmichael
Copy link
Owner

What am I doing wrong ?

22:32:29 ~/src/hledger$ stack exec -- hledger --version
hledger 1.22-84-g41ce169ba, mac-x86_64
22:32:34 ~/src/hledger$ stack exec -- hledger bal -f examples/sample.journal
                  $1  assets:bank:saving
                 $-2  assets:cash
                  $1  expenses:food
                  $1  expenses:supplies
                 $-1  income:gifts
                 $-1  income:salary
                  $1  liabilities:debts
--------------------
                   0  
22:32:42 ~/src/hledger$ stack exec -- hledger bal -f examples/sample.journal --commodity-column
                  $1  assets:bank:saving
                 $-2  assets:cash
                  $1  expenses:food
                  $1  expenses:supplies
                 $-1  income:gifts
                 $-1  income:salary
                  $1  liabilities:debts
--------------------
                   0  

@0xlwu
Copy link
Contributor Author

0xlwu commented Jul 28, 2021

I see, thanks. We should improve the UX here. Minimising the number of special cases helps users, so ideally this would not be limited to periodic reports ?

I don't particularly mind either way we handle it. In case we do want to render the single-period commodity-column differently, would you prefer %(total) %(commodity) %(account), or perhaps %(account) %(commodity) %(total)?

If it must be so, the help and manual should say so and it should give an error in that case, like with --tree mode - it's less confusing.

For tree mode, commodity-column still does the 'right thing' (see mockups #1626 (comment)). I think the main issue is that it is verbose.

renderTableB topts fr fc f = renderTableB' topts (fmap fc) (\(rh, as) -> (fr rh, fmap f as))

-- | A version of renderTable that operates on rows
renderTableB' :: TableOpts -- ^ Options controlling Table rendering
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should use a more understandable name here, something like renderTableByRowsB'.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And perhaps a more detailed haddock.
A version of renderTableB which uses a different rendering function for each row.

@Xitian9
Copy link
Collaborator

Xitian9 commented Jul 30, 2021

I've taken a look at the remaining code, and it looks great! It's a bit disappointing that budget reports require so much extra code to handle, but they are a bit clunky in any case. I'm sure there's a more principled solution to budget reports in general which will require less cludgy code for many things.

If we feel that a separate commodity column should also be put in for single-column balance reports, I think your proposal looks good to me.

         5120.50  USD  Assets:US:ETrade:Cash
           70.00  GLD  Assets:US:ETrade:GLD
           17.00  ITOT Assets:US:ETrade:ITOT
           36.00  VEA  Assets:US:ETrade:VEA
          294.00  VHT  Assets:US:ETrade:VHT

@0xlwu
Copy link
Contributor Author

0xlwu commented Jul 30, 2021

It's a bit disappointing that budget reports require so much extra code to handle, but they are a bit clunky in any case. I'm sure there's a more principled solution to budget reports in general which will require less cludgy code for many things.

Agreed. I tried to get something working cell-wise (by having total, left-bracket, percent, %-delim, budget, right-bracket be separate cells) so that alignment / width would be solved by renderTable* , but relatively quickly ran into issues trying to align the header columns with that.

If we feel that a separate commodity column should also be put in for single-column balance reports, I think your proposal looks good to me.

Sounds good, I'll take a look tomorrow. Thanks!

@Xitian9
Copy link
Collaborator

Xitian9 commented Jul 30, 2021

Agreed. I tried to get something working cell-wise (by having total, left-bracket, percent, %-delim, budget, right-bracket be separate cells) so that alignment / width would be solved by renderTable* , but relatively quickly ran into issues trying to align the header columns with that.

Yes, I tried to do that at some point too, and that was exactly the problem I ran into. The ideal solution would be to allow headers to have a column width argument, but this would require re-implementing even more of the Text.Tabular structure for our own purposes.

@simonmichael
Copy link
Owner

simonmichael commented Jul 30, 2021 via email

@0xlwu 0xlwu force-pushed the commodity-account branch from a8e13ff to 99e09c5 Compare July 30, 2021 16:23
@0xlwu
Copy link
Contributor Author

0xlwu commented Jul 30, 2021

For ease of implementation, the single-period balance report in --commodity-column does not respect custom formatting.

@0xlwu
Copy link
Contributor Author

0xlwu commented Aug 5, 2021

@Xitian9 @simonmichael What are your thoughts on the PR at this point? Would it be helpful if I rebased/squashed commits again?

@simonmichael simonmichael added the needs-review To unblock: needs more code/docs/design review by someone label Aug 5, 2021
@simonmichael
Copy link
Owner

Thanks, I was prioritising other things, I'll check this again soon.

@Xitian9
Copy link
Collaborator

Xitian9 commented Aug 6, 2021

It's overall looking very good. I'll be very happy to see this merged soon.

I'll do a proper code review later, but here are some quick suggestions:

  • If it's not too hard, squash together any commits which involve making changes and then undoing them later (e.g. creating and then deleting showMixedAmountLinesB)
  • Create a test for normal (non-multicolumn) balance report

0xlwu added 5 commits August 6, 2021 08:11
This adds the `--commodity-column` option that displays each commodity
on a separate line and the commodities themselves as a separate column.
The initial design considerations are at
simonmichael.hledger.issues.1559

The single-period balance report with `--commodity-column` does not
interoperate with custom formats.
Extension of commodity-column to budget reporting.
Add documentation and sample output for `--commodity-column` behavior
and functional tests e.g single-period balance, yearly balance, and
yearly budget
Makes them consistent with the remaining cells and fixes awkward
alignment issue in commodity-column mode where we don't display
anything
Previously would not actually display anything since the Cell's
WideBuilders are single-element list. Just dispatch to showMixedAmountB
to do the right thing.
@0xlwu 0xlwu force-pushed the commodity-account branch from 99e09c5 to aefdaaf Compare August 6, 2021 13:28
@Xitian9
Copy link
Collaborator

Xitian9 commented Aug 8, 2021

Looks good to me! I have no particular code comments.

@simonmichael simonmichael removed the needs-review To unblock: needs more code/docs/design review by someone label Aug 8, 2021
@simonmichael
Copy link
Owner

Thanks for the review. Looking good @la-wu! I have some minor comments about docs, if you don't mind I'm going to merge and just do them.

I think this will be a valuable feature, unlocking charting for one thing. Thank you!

@simonmichael simonmichael merged commit 1be74a0 into simonmichael:master Aug 8, 2021
simonmichael added a commit that referenced this pull request Aug 8, 2021
- promote the heading one level
- periodic CSV reports are supported too
- slight edits to manual and flag description
@simonmichael
Copy link
Owner

@simonmichael
Copy link
Owner

@la-wu : in the doc, the -T in the CSV example looks redundant. More importantly I couldn't reproduce your example, trying the below; what am I missing ?

$ hledger -f examples/bcexample.hledger bal -T -O csv etrade
"account","balance"
"Assets:US:ETrade:Cash","5120.50 USD"
"Assets:US:ETrade:GLD","70.00 GLD"
"Assets:US:ETrade:ITOT","17.00 ITOT"
"Assets:US:ETrade:VEA","36.00 VEA"
"Assets:US:ETrade:VHT","294.00 VHT"
"Income:US:ETrade:Gains","-373.34 USD"
"total","70.00 GLD, 17.00 ITOT, 4747.16 USD, 36.00 VEA, 294.00 VHT"

@simonmichael
Copy link
Owner

Ah, found it: -T should be -3. Fixed.

@0xlwu
Copy link
Contributor Author

0xlwu commented Aug 8, 2021

All the examples were based on -f examples/bcexample.hledger assets.*etrade -3 (which are the flags used in the mockups on the original issue).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-WISH Some kind of improvement request or proposal. balance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants