Dry Goods | What is UTXO Merger?

Source: BixinInstitute , original title "UTXO Merger"

By David A. Harding

This article was originally compiled by the Coin Trust Research Institute, the original link:

https://github.com/bitcoinops/scaling-book/blob/6a7b1e9cc64ddf832b86276e7569604334f9012c/x.consolidation/consolidation.md

• 5 consecutive years of decline, can BTC break the January curse?
• Six major central banks such as Britain, Japan and Europe hold a group to explore fiat digital currencies, China is temporarily in a leading position
• Academician Rong Chunming: The blockchain system is still based on U.S. China or overtaking on curved roads

The copyright of this article belongs to the original author, which only represents the author's own opinion, and does not represent the opinion or position of Cointrust or Cointrust Research Institute.

This article is about 6500 words and it takes about 20 minutes to read the full text.

Unused transaction output (UTXO) consolidation refers to the consolidation of the value of Bitcoins stored in multiple UTXO into one UTXO. This can reduce the number of UTXOs you control while keeping your balance roughly the same, so you have a better chance of creating smaller, more affordable transactions when you need to pay for a quick confirmation.

Merge example

Think of a transaction as a rectangle whose height is its rate, its length is its size (block weight), and its area is its total cost (rate multiplied by size). For example, the following is a simple transaction that costs from one P2WPKH UTXO (input) to two P2WPKH outputs-one output to the customer and the other back to the consumer as change.

Most wallets end up with situations where they don't have enough value in a UTXO to pay the amount they want to spend. This requires them to add more value to the transaction by spending more UTXO. For example, let's spend two additional P2WPKH UTXOs in our sample transaction:

Although this may seem like a small change, it almost doubled our transaction size and cost. This is where mergers come into play. By preparing in advance, we can merge three UTXOs for large transactions into one UTXO through a low-rate transaction that we send ourselves. We can then use a single UTXO to pay the customer in a high-fee transaction in the same form as the first example transaction above:

Comparing the total cost, we found that the two-transaction version was 40% cheaper than a single high-rate transaction using three UTXOs. This is possible even if the two-transaction version uses more total block space.

If the merge transaction is confirmed before you send the second transaction to the customer, then the customer does not experience any additional delay compared to a single transaction, so you can save a small amount of money without changing the recipient ’s Experience.

UTXO spent on average

From the previous example, we can see that when multiple UTXOs need to be used in a transaction, the expenditure is inefficient and the merger is useful. How often does this happen to you? If you have access to the following two data, you can use the simple formula provided below to estimate the average amount of UTXO used per transaction.

Incoming-The amount of output your wallet gets from someone else.

Outgoing-The number of transactions created by the wallet to send Bitcoin to others. If you use payment batching (https://github.com/bitcoinops/scaling-book/blob/master/x.payment_batching/payment_batching.md), please note that this is the number of transactions, not the number of payments. To improve accuracy, you can include in this total the number of transactions you expect to make before the wallet is empty (that is, your current wallet balance divided by the average transfer of transactions).

To calculate the average amount of UTXO spent per transaction, use the following formula:

average_utxos = (transfer in + transfer out) / transfer out

The reason why the roll-out is both above and below the division sign is because we expect that most transactions you create will also generate a change output, returning the unused value to your wallet as a new UTXO.

The minimum realistic result of the above equation is 1.0, which is also the minimum amount of UTXO that a transaction can contain. The higher your results are above 1.0, the more you will benefit from the UTXO merge, which will be seen in the next section.

Expected savings

To estimate how much you can save by using merges, let's see how the total cost of a typical transaction depends on how much UTXO it spends (that is, how much it increases in size). When viewing this chart, compare the x-axis to the average number of UTXOs you calculated in the previous section.

What if we could use a merge transaction that pays only 1/10 of the rate used in the picture above and merge all the extra UTXO into one UTXO? We can then use a single UTXO to create regular expenses at the full rate. Overlaying this data at the top of the image above, we can see how much can be saved:

Let's plot the savings as a percentage:

Obviously, the actual amount we save will depend on the ratio between the low rate used for consolidation and the higher rate used to pay our customers. Using the data provided in the appendix to this article, at the time of writing, the following ratios can usually be achieved:

Another thing that will affect our savings is how much data we need to add to the transaction in order to spend each UTXO; that is, the size of the transaction input. For P2WPKH, it is approximately 68 vbytes per UTXO; but for P2SH 2-3 multi-signs, it is approximately 294 vbytes. Assuming we unify the rate of regular expenses to 1/10, the savings of various common script templates are:

Based on the above estimates, we expect that, at least some of the time, almost all active Bitcoin services can save a lot of money through consolidation. Exceptions are that the average number of UTXOs is very small, or services that try to merge when the difference between fast (high) and slow (low) rates continues to be small.

UTXO pool

The example above assumes that you will change from having n UTXOs to only one UTXO, but if you need to broadcast a transaction and one of your transactions is not confirmed, then this may cause problems.

From the perspective of a full node (such as Bitcoin Core), UTXO comes in two forms: confirmed and unconfirmed. It has been confirmed that UTXO is the output of a transaction on the best blockchain. Most existing nodes will always accept confirmed UTXO spending at first glance, provided that the spending is valid, that is, paying the appropriate rate, less than 100,000 vbytes, and following some other simple rules.

An unconfirmed UTXO is the output of a transaction that is still in the memory pool (that is, not yet confirmed). To prevent DoS attacks against a node's memory pool, the node places additional restrictions on transactions that use unconfirmed UTXO. This means that you cannot guarantee the use of unconfirmed UTXO at any time. For example, until the first transaction (tx0) is confirmed, the wallet may not be able to propagate the second transaction (tx1) shown below:

Because many users want to be notified of new unconfirmed transactions in their wallets within seconds of requesting a payment (this requires nodes to accept and relay these transactions immediately), many spenders do not want to fall into this state themselves : Waiting for a previous transaction to be confirmed, but cannot submit a new transaction to the network. This will cause these consumers to maintain a UTXO pool-a set of confirmed UTXOs that can be used at any time.

UTXO pools do not significantly affect the savings described earlier because you can always wait until later to merge the pools into one UTXO. However, the more times you allocate funds between different UTXOs to ensure immediate spendability, the more likely you will need to pay beyond the value of any single UTXO you control-this weakens (or may completely offset ) Cost savings from UTXO merger.

If you rarely encounter this problem in the UTXO pool, you can ignore this situation because it is inefficient. If it happens often, there are several options:

1. Increase the average value of the output in the UTXO pool by keeping the number of outputs the same and increasing the value of each output, thereby increasing the amount of funds in the pool as a whole. Please note that this will also increase your monetary loss due to theft or hot wallet accident.

2. Increase the average value of the output in the UTXO pool by keeping the total amount unchanged but reducing the number of outputs. You can do this with bulk payments to reduce the number of UTXOs required at any given time.

3. As described later in this chapter, focus on creating transactions that do not change. For reasons described in the following sections, transactions that do not have change are generally not as effective as mergers, but they can save you some money. When optimizing your UTXO pool for constant transactions, you want UTXO to have various amounts so that you have as many different combinations of amounts as possible.

Optimal merge size

To merge UTXO, the data you want to include in the transaction is not only UTXO, but also the output, which returns the funds to yourself, and contains some transaction templates (version number, nLockTime, etc.). The more UTXOs you combine in a single transaction, the better the distribution of the fixed costs of that transaction among UTXOs, and the cost of each UTXO asymptotically approaches the average size of one of the inputs.

As shown above, the amount of savings saved by merging additional UTXOs at the same time is not particularly large, so you can get reasonable efficiency even if you don't pay attention to the size of the merging.

Cold wallet merger

Some services (such as certain exchanges) have a basic division of responsibilities between their wallets. Received customer deposits or payments will be deposited into a cold wallet. Payments or withdrawals are made using hot wallets. When necessary, one or more staff members transfer money from a cold wallet to a hot wallet.

A simple merge strategy is to put all UTXO received in a certain period (such as a week) into a cold wallet, and then merge them into one UTXO. However, you may be wondering if this is wasteful because you may eventually split a single UTXO into multiple UTXOs in the hot wallet UTXO pool in the future. For example, if you merge 10 UTXO into 1 UTXO, it seems inefficient to transfer it back to 5 UTXO.

However, the two alternative options may be worse. The first alternative is that you don't merge cold wallets. You just have to wait until you need to transfer to the hot wallet and perform the merge. This minimizes the use of blockchain space-but all of this space is used in a single large transaction. If you need a transaction for quick confirmation–for example, because you need the money to move a hot wallet–you may pay a high fee for the transaction, losing the benefits of consolidating cold wallets.

The second alternative is to merge 10 UTXO into 5 UTXO in the cold wallet, and then transfer 5 cold UTXO to 5 hot wallet UTXO. Of all the available options, this will use the most block space. It created a large transaction at a low consolidation rate and a medium-sized transaction at a higher rate to send money to a hot wallet. This makes it the worst performing option.

Although it might seem silly to merge UTXO to the maximum and then split later, the simple merge strategy described at the beginning of this section does provide the greatest savings.

Private issues

When you receive a payment for a new address that has never been used, the analyst looking at the history of the blockchain will not know who owns the UTXO. However, if you merge UTXO with other UTXOs that they have determined to belong to you, the analyst can make reasonable assumptions that all of the combined funds belong to you.

Therefore, users trying to maximize the privacy of their Bitcoin balance or transaction history should not use the UTXO merger described in this document. Users who have previously used technologies such as coinjoin to mix their UTXO ownership history with other UTXOs should especially avoid this.

It is possible to perform a limited amount of consolidation during coin mixing, for example, in a Joinmarket or Wasabi implementation. In addition, ideas for future protocol development, such as channel factories (http: // devbcop / en / topics / channel-factories /) or hybrid pools (joinpools, https://freenode.irclog.whitequark.org/bitcoin- wizards / 2019-05-21 # 24639483;), it may make it more common for multiple UTXOs owned by multiple people to be combined into one UTXO and then used for periodic expenditures, making it less clear for analysts Merged currency.

Select UTXO to merge

Your goal is to make as many payments as possible with one UTXO, so you should first merge the lowest value UTXO. In addition, you can reduce the loss of privacy by ensuring that every time you choose to receive UTXO for a specific address, you also choose to receive all other UTXO for the same address. For example, suppose you want to merge three UTXOs in the following example wallet:

You should choose abcdef: 0 because it is the lowest value UTXO. Then select cdefab: 0 because it is sent to the same address as abcdef: 0. Finally, choose bcdefa: 0 because it is the second lowest value UTXO.

For more information on selecting the relevant UTXO, see (https://bitcoinops.org/en/newsletters/2018/07/31/#bitcoin-core-12257) -avoidpartialspend configuration option in Bitcoin Core.

Merging and competing technologies

While using UTXO merges, other cost-saving techniques cannot be effectively used, so the following sections compare them to merges, highlighting their advantages and disadvantages.

No change

Non-change transactions do not include change output, which reduces the size of a typical transaction by approximately 31 to 43 vbytes. Although it is possible to create a non-changing transaction when only one UTXO is spent, which is the target of a merger, it usually takes two or more UTXOs to produce a non-changing transaction. This is because if you are willing to use any number of inputs, the number of UTXO combinations that can be included in a transaction can greatly help in finding a non-changing transaction.

Non-changing transactions save block space and costs, so it is highly recommended that you use them. However, effectively using UTXO to merge can almost always save more money than a no-change transaction that requires two or more UTXO to be spent in the same transaction. This is because the transaction input (UXTO cost) of all commonly used scripts is much larger than the transaction output, so the cost saved by omitting the output does not make up for the cost of having to include additional input.

Non-changing transactions will also delete a UTXO from your wallet, just as the UTXO merger reduces your future transaction burden. The money you save is harder to analyze, because usually you round up transaction fees to create non-changing transactions, which increases the expense rate. However, even if we ignore this, because the input is much larger than the output, merging at a lower rate will still save more money than creating a two-input no-change transaction with a higher expense rate. Consider an example: Alice and Bob each receive 10 UTXOs. Alice consolidates her UTXO into a single UTXO at a cost of 1/10 of the normal expense and sends five payments. Bob keeps his UTXO separately and sends five payments himself, each using two UTXOs to create a non-changing transaction at the normal spending rate:

Alice's combined transaction (ten inputs and one output) is 679.25 vbytes; her subsequent five payment transactions (one input and two outputs) are 136.50 vbytes. Bob's five non-changing transactions (two inputs and one output each) are 169.25 vbytes in size.

If Alice and Bob both send their payment transactions at the same rate, we can use some simple algebra to calculate the percentage percentage of Bob's rate required for a merge transaction to equal the money saved by Alice using a non-changing transaction:

719.25 * x + 5 * 140.50 = 5 * 177.25

x = 0.26 (rounded)

In short, if Alice can merge at a quarter or less of what she normally spends, she will save more money than Bob, even if she uses nearly twice as much block space as he does. The rate data in the appendix to this chapter shows that, under our rough assumptions, it is possible to save more than 96% of the time.

If Bob needs to use more than two inputs for each transaction to create a non-changing transaction, even if Alice uses a correspondingly higher fee rate for the combined transaction, she can save even more money.

That is, the obvious advantage of non-changing transactions over merged transactions is that they allow spenders to save block space and modest transaction fees, while retaining their privacy in case of the "merged privacy" section above The problem. For most individual consumers, even small businesses that spend less frequently, privacy advantages are more important to them than the transaction fees saved by mergers.

Predictive / Organic UTXO Management

Predictive UTXO management is a service provided by BitGo: "Reduce the overall cost by reducing transaction size at high rates while automatically clearing and processing small fragments of coins at low transaction fees." (Https: // www. businesswire.com/news/home/20180726005483/en/BitGo-Introduces-Predictive-UTXO-Management-Delivering-30) The specific implementation of this technology is proprietary, but it may be similar to the organic UTXO merger described by Dmitry Petuhov (Https://www.reddit.com/r/Bitcoin/comments/9kxn01/organic_utxo_consolidation_in_the_course_of/).

Such technologies eliminate the extra blockchain space used by merger transactions, making them potentially more efficient and saving more transaction fees. It may be easier to implement certain services because there is no need to wait for hours or days to confirm a consolidated transaction. In addition, BitGo's predictive UTXO management system, in particular, seems to be able to achieve partial savings using the non-changing transactions described in the previous section, as well as the additional savings and privacy benefits described therein.

However, in terms of cost savings, for those who wish to confirm their spending transactions in the next few blocks, UTXO consolidation usually saves more money than predictive / organic technologies. This is because the tens of bytes saved in the blockchain space by using predictive / organic transactions do not match the ability to save a large percentage of UTXO expenditure costs by spending UTXO at lower rates. In fact, this is quantified in the previous illustration:

For people who spend an average of two or more UTXOs per transaction and can be merged at 50% or less of the normal rate (and those who use P2WPKH transactions analyzed here), the merger can at least bring the cost to the end Balanced and can easily save more money than sending ordinary transactions. The rate data included in the appendix to this chapter shows that during our analysis of November and December 2019, there was a 99.85% probability of saving at least 50% of the rate for the nominal 2 block confirmation targets.

Similarly, the predictive / organic approach does have the advantage that its implementation may be much easier compared to the consolidation of existing services, and when combined with non-changing transactions, can achieve modest additional savings and enhance privacy . However, spenders focused on maximizing cost savings may want to focus on UTXO mergers.

Increase transaction fees

The biggest savings in consolidated transactions should be sent at the lowest rate you think allows transactions to be confirmed before they are needed. For example, if you can wait for a week to confirm a transaction, you should use the transaction fee estimation tool's target rate to confirm within 1008 blocks.

However, to make a previously-confirmed merge transaction faster to confirm, or because of unexpected increases in rates, sometimes you may need to increase its fees. In this case, you have several options:

For a merge from a hot wallet to a hot wallet (or a merge from a hot wallet to a cold wallet), you should issue the BIP125 (https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki) signal, In this way, you can use Replace-by-Fee (RBF) to increase transaction fees, which is the most effective type of increasing transaction fees.

For merges sent from cold wallets to hot wallets, you can easily use Child-Pays-For-Parent (CPFP) to increase transaction fees.

For merges sent from a cold wallet to a cold wallet, you may need to design your transactions so that they always contain a small amount of output paid to the hot wallet. This way, you can use the CPFP billing feature with a hot wallet without having to reopen the cold wallet. However, unless you plan to transfer money to your hot wallet, this extra output may slightly reduce the efficiency of the merger.

in conclusion

For organizations that receive and send payments frequently, consolidation is one of the most effective ways to save money. It is relatively easy to implement, and if done well, the person receiving your payment will never notice that you are doing so.

See also

Anthony Towns' Field Report: Consolidating 4 Million UTXOs in Xapo (https://bitcoinops.org/en/xapo-utxo-consolidation/)

Mark Erhardt's "Speech: Unspent Management and the Choice of Coins" (https://www.youtube.com/watch?v=hrlNN3BSB6w&feature=emb_logo)

Appendix A:

The following figure shows the estimated savings of 2 block confirmation goals (about 20 minutes) and 1008 block confirmation goals (about 1 week) estimated using Bitcoin Core's transaction fees, over a period of time.

The table below summarizes how often certain savings are achieved. We see that it is almost always possible to save 50% of the rate by waiting for a week, and usually 90% or more.

We use the aforementioned one-week goal because this is the longest term provided by Bitcoin Core's evaluator, and for many organizations, it is easy to wait a week for a consolidated transaction to confirm. Subsequent sections on rate estimation and emergency trade-offs will analyze the rate differences between different confirmation targets in more detail.

-The End-

We will continue to update Blocking; if you have any questions or suggestions, please contact us!