149da42 db_lmdb: enable batch transactions by default (stoffu)
34cb6b4 add --regtest and --fixed-difficulty for regression testing (vicsn)
9e1403e update get_info RPC and bump RPC version (vicsn)
207b66e first new functional tests (vicsn)
This gets rid of the temporary precalc cache.
Also make the RPC able to send data back in binary or JSON,
since there can be a lot of data
This bumps the LMDB database format to v3, with migration.
on_generateblocks RPC call combines functionality from the on_getblocktemplate and on_submitblock RPC calls to allow rapid block creation. Difficulty is set permanently to 1 for regtest.
Makes use of FAKECHAIN network type, but takes hard fork heights from mainchain
Default reserve_size in generate_blocks RPC call is now 1. If it is 0, the following error occurs 'Failed to calculate offset for'.
Queries hard fork heights info of other network types
f8dd433 epee: fix detection of 172.16.0.0/172.31.255.255 local IP range (moneromooo-monero)
5db9e3c unit_tests: add tests for local IP range detection (moneromooo-monero)
db55263 threadpool: allow constructing an object, and misc tweaks (moneromooo-monero)
ce173cb core: remove threadpool dependency from header (moneromooo-monero)
3147468 unit_tests: add threadpool unit test (moneromooo-monero)
a connection's timeout is halved for every extra connection
from the same host.
Also keep track of when we don't need to use a connection
anymore, so we can close it and free the resource for another
connection.
Also use the longer timeout for non routable local addresses.
0e7ad2e2 Wallet API: generalize 'bool testnet' to 'NetworkType nettype' (stoffu)
af773211 Stagenet (stoffu)
cc9a0bee command_line: allow args to depend on more than one args (stoffu)
55f8d917 command_line::get_arg: remove 'required' for dependent args as they're always optional (stoffu)
450306a0 command line: allow has_arg to handle arg_descriptor<bool,false,true> #3318 (stoffu)
9f9e095a Use `genesis_tx` parameter in `generate_genesis_block`. #3261 (Jean Pierre Dudey)
The basic approach it to delegate all sensitive data (master key, secret
ephemeral key, key derivation, ....) and related operations to the device.
As device has low memory, it does not keep itself the values
(except for view/spend keys) but once computed there are encrypted (with AES
are equivalent) and return back to monero-wallet-cli. When they need to be
manipulated by the device, they are decrypted on receive.
Moreover, using the client for storing the value in encrypted form limits
the modification in the client code. Those values are transfered from one
C-structure to another one as previously.
The code modification has been done with the wishes to be open to any
other hardware wallet. To achieve that a C++ class hw::Device has been
introduced. Two initial implementations are provided: the "default", which
remaps all calls to initial Monero code, and the "Ledger", which delegates
all calls to Ledger device.
6d8b29ef fix some link errors in debug mode for macos (stoffu)
fdd4c5e5 move memwipe to epee to avoid common<->crypto circular dependencies (moneromooo-monero)
40ab12a7 epee: remove dependency on common (moneromooo-monero)
bd5cce07 network_throttle: fix ineffective locking (moneromooo-monero)
e0a61299 network_throttle: remove unused xxx static member (moneromooo-monero)
24f584d9 cryptonote_core: remove unused functions with off by one bugs (moneromooo-monero)
b1634aa3 blockchain: don't leave dangling pointers in this (moneromooo-monero)
8e60b81c cryptonote_core: fix db leak on error (moneromooo-monero)
213e326c abstract_tcp_server2: log init_server errors as fatal (moneromooo-monero)
b51dc566 use const refs in for loops for non tiny types (moneromooo-monero)
f0568ca6 net_parse_helpers: fix regex error checking (moneromooo-monero)
b49ddc76 check accessing an element past the end of a container (moneromooo-monero)
2305bf26 check return value for generate_key_derivation and derive_public_key (moneromooo-monero)
a4240d9f catch const exceptions (moneromooo-monero)
45a1c4c0 add empty container sanity checks when using front() and back() (moneromooo-monero)
56fa6ce1 tests: fix a buffer overread in a unit test (moneromooo-monero)
b4524892 rpc: guard against json parsing a non object (moneromooo-monero)
c2ed8618 easylogging++: avoid buffer underflow (moneromooo-monero)
187a6ab2 epee: trap failure to parse URI from request (moneromooo-monero)
061789b5 checkpoints: trap failure to load JSON checkpoints (moneromooo-monero)
ba2fefb9 checkpoints: pass std::string by const ref, not const value (moneromooo-monero)
38c8f4e0 mlog: terminate a string at last char, just in case (moneromooo-monero)
d753d716 fix a few leaks by throwing objects, not newed pointers to objects (moneromooo-monero)
fe568db8 p2p: use size_t for arbitrary counters instead of uint8_t (moneromooo-monero)
46d6fa35 cryptonote_protocol: sanity check chain hashes from peer (moneromooo-monero)
25584f86 cryptonote_protocol: print peer versions when unexpected (moneromooo-monero)
490a5d41 rpc: do not try to use an invalid txid in relay_tx (moneromooo-monero)
Scheme by luigi1111:
Multisig for RingCT on Monero
2 of 2
User A (coordinator):
Spendkey b,B
Viewkey a,A (shared)
User B:
Spendkey c,C
Viewkey a,A (shared)
Public Address: C+B, A
Both have their own watch only wallet via C+B, a
A will coordinate spending process (though B could easily as well, coordinator is more needed for more participants)
A and B watch for incoming outputs
B creates "half" key images for discovered output D:
I2_D = (Hs(aR)+c) * Hp(D)
B also creates 1.5 random keypairs (one scalar and 2 pubkeys; one on base G and one on base Hp(D)) for each output, storing the scalar(k) (linked to D),
and sending the pubkeys with I2_D.
A also creates "half" key images:
I1_D = (Hs(aR)+b) * Hp(D)
Then I_D = I1_D + I2_D
Having I_D allows A to check spent status of course, but more importantly allows A to actually build a transaction prefix (and thus transaction).
A builds the transaction until most of the way through MLSAG_Gen, adding the 2 pubkeys (per input) provided with I2_D
to his own generated ones where they are needed (secret row L, R).
At this point, A has a mostly completed transaction (but with an invalid/incomplete signature). A sends over the tx and includes r,
which allows B (with the recipient's address) to verify the destination and amount (by reconstructing the stealth address and decoding ecdhInfo).
B then finishes the signature by computing ss[secret_index][0] = ss[secret_index][0] + k - cc[secret_index]*c (secret indices need to be passed as well).
B can then broadcast the tx, or send it back to A for broadcasting. Once B has completed the signing (and verified the tx to be valid), he can add the full I_D
to his cache, allowing him to verify spent status as well.
NOTE:
A and B *must* present key A and B to each other with a valid signature proving they know a and b respectively.
Otherwise, trickery like the following becomes possible:
A creates viewkey a,A, spendkey b,B, and sends a,A,B to B.
B creates a fake key C = zG - B. B sends C back to A.
The combined spendkey C+B then equals zG, allowing B to spend funds at any time!
The signature fixes this, because B does not know a c corresponding to C (and thus can't produce a signature).
2 of 3
User A (coordinator)
Shared viewkey a,A
"spendkey" j,J
User B
"spendkey" k,K
User C
"spendkey" m,M
A collects K and M from B and C
B collects J and M from A and C
C collects J and K from A and B
A computes N = nG, n = Hs(jK)
A computes O = oG, o = Hs(jM)
B anc C compute P = pG, p = Hs(kM) || Hs(mK)
B and C can also compute N and O respectively if they wish to be able to coordinate
Address: N+O+P, A
The rest follows as above. The coordinator possesses 2 of 3 needed keys; he can get the other
needed part of the signature/key images from either of the other two.
Alternatively, if secure communication exists between parties:
A gives j to B
B gives k to C
C gives m to A
Address: J+K+M, A
3 of 3
Identical to 2 of 2, except the coordinator must collect the key images from both of the others.
The transaction must also be passed an additional hop: A -> B -> C (or A -> C -> B), who can then broadcast it
or send it back to A.
N-1 of N
Generally the same as 2 of 3, except participants need to be arranged in a ring to pass their keys around
(using either the secure or insecure method).
For example (ignoring viewkey so letters line up):
[4 of 5]
User: spendkey
A: a
B: b
C: c
D: d
E: e
a -> B, b -> C, c -> D, d -> E, e -> A
Order of signing does not matter, it just must reach n-1 users. A "remaining keys" list must be passed around with
the transaction so the signers know if they should use 1 or both keys.
Collecting key image parts becomes a little messy, but basically every wallet sends over both of their parts with a tag for each.
Thia way the coordinating wallet can keep track of which images have been added and which wallet they come from. Reasoning:
1. The key images must be added only once (coordinator will get key images for key a from both A and B, he must add only one to get the proper key actual key image)
2. The coordinator must keep track of which helper pubkeys came from which wallet (discussed in 2 of 2 section). The coordinator
must choose only one set to use, then include his choice in the "remaining keys" list so the other wallets know which of their keys to use.
You can generalize it further to N-2 of N or even M of N, but I'm not sure there's legitimate demand to justify the complexity. It might
also be straightforward enough to support with minimal changes from N-1 format.
You basically just give each user additional keys for each additional "-1" you desire. N-2 would be 3 keys per user, N-3 4 keys, etc.
The process is somewhat cumbersome:
To create a N/N multisig wallet:
- each participant creates a normal wallet
- each participant runs "prepare_multisig", and sends the resulting string to every other participant
- each participant runs "make_multisig N A B C D...", with N being the threshold and A B C D... being the strings received from other participants (the threshold must currently equal N)
As txes are received, participants' wallets will need to synchronize so that those new outputs may be spent:
- each participant runs "export_multisig FILENAME", and sends the FILENAME file to every other participant
- each participant runs "import_multisig A B C D...", with A B C D... being the filenames received from other participants
Then, a transaction may be initiated:
- one of the participants runs "transfer ADDRESS AMOUNT"
- this partly signed transaction will be written to the "multisig_monero_tx" file
- the initiator sends this file to another participant
- that other participant runs "sign_multisig multisig_monero_tx"
- the resulting transaction is written to the "multisig_monero_tx" file again
- if the threshold was not reached, the file must be sent to another participant, until enough have signed
- the last participant to sign runs "submit_multisig multisig_monero_tx" to relay the transaction to the Monero network
43f5269f Wallets now do not depend on the daemon rpc lib (moneromooo-monero)
bb89ae8b move connection_basic and network_throttle from src/p2p to epee (moneromooo-monero)
4abf25f3 cryptonote_core does not depend on p2p anymore (moneromooo-monero)
3dffe71b new wipeable_string class to replace std::string passphrases (moneromooo-monero)
7a2a5741 utils: initialize easylogging++ in on_startup (moneromooo-monero)
54950829 use memwipe in a few relevant places (moneromooo-monero)
000666ff add a memwipe function (moneromooo-monero)
0d9c0db9 Do not build against epee_readline if it was not built (Howard Chu)
178014c9 split off readline code into epee_readline (moneromooo-monero)
a9e14a19 link against readline only for monerod and wallet-wallet-{rpc,cli} (moneromooo-monero)
437421ce wallet: move some scoped_message_writer calls from the libs (moneromooo-monero)
e89994e9 wallet: rejig to avoid prompting in wallet2 (moneromooo-monero)
ec5135e5 move input_line from command_line to simplewallet (moneromooo-monero)
082db75f move cryptonote command line options to cryptonote_core (moneromooo-monero)
This patch allows to filter out sensitive information for queries that rely on the pool state, when running in restricted mode.
This filtering is only applied to data sent back to RPC queries. Results of inline commands typed locally in the daemon are not affected.
In practice, when running with `--restricted-rpc`:
* get_transaction_pool will list relayed transactions with the fields "last relayed time" and "received time" set to zero.
* get_transaction_pool will not list transaction that have do_not_relay set to true, and will not list key images that are used only for such transactions
* get_transaction_pool_hashes.bin will not list such transaction
* get_transaction_pool_stats will not count such transactions in any of the aggregated values that are computed
The implementation does not make filtering the default, so developers should be mindful of this if they add new RPC functionality.
Fixes#2590.
ff7745bb Edited test readme for accuracy and depth (Cole Lightfighter)
c300ae56 Added test documentation & Keccak unit test (Cole Lightfighter)
f6119a8e Added test documentation & Keccak unit test (Cole Lightfighter)
Tests for checking proper error throwing for out-of-bounds subaddress
indexes, and proper addition of subaddresses.
Signed-off-by: Cole Lightfighter <cole@onicsla.bz>
The commands handler must not be destroyed before the config
object, or we'll be accessing freed memory.
An earlier attempt at using boost::shared_ptr to control object
lifetime turned out to be very invasive, though would be a
better solution in theory.
- internal nullptr checks
- prevent modifications to network_address (shallow copy issues)
- automagically works with any type containing interface functions
- removed fnv1a hashing
- ipv4_network_address now flattened with no base class
6137a0b9 blockchain: reject unsorted ins and outs from v7 (moneromooo-monero)
16afab90 core: sort ins and outs key key image and public key, respectively (moneromooo-monero)
0c36b9f9 common: add apply_permutation file and function (moneromooo-monero)
It was always returning true, and could not be foreseen to
usefully return errors in the future. This silences CID 162652
as well as saves some checking code in a few places.
And optimize import startup:
Remember start_height position during initial count_blocks pass
to avoid having to reread entire file again to arrive at start_height
If monerod is started with default sync mode, set it to SAFE after
synchronization completes. Set it back to FAST if synchronization
restarts (e.g. because another peer has a longer blockchain).
If monerod is started with an explicit sync mode, none of this
automation takes effect.
Add get_fork_version and add_ideal_fork_version to core so
cryptonote_protocol does not have to need the Blockchain
class directly, as it's not in its dependencies, and add
those to the fake core classes in tests too.
A block queue is now placed between block download and
block processing. Blocks are now requested only from one
peer (unless starved).
Includes a new sync_info coommand.
- Performance improvements
- Added `span` for zero-copy pointer+length arguments
- Added `std::ostream` overload for direct writing to output buffers
- Removal of unused `string_tools::buff_to_hex`
Minimum mixin 4 and enforced ringct is moved from v5 to v6.
v5 is now used for an increased minimum block size (from 60000
to 300000) to cater for larger typical/minimum transaction size.
The fee algorithm is also changed to decrease the base per kB
fee, and add a cheap tier for those transactions which we do
not care if they get delayed (or even included in a block).
a427235e core: add a missing newline on a string to be logged (moneromooo-monero)
b6a2230e unit_tests: fix minor blockchain_db regression (moneromooo-monero)
c488eca5 hardfork: tone down some logs (moneromooo-monero)
- fix wrong block being used when a new block is received between
a node elaying a fluffy block and sending a new fluffy block
with txes a peer did not have
- misc a neverending ping pong requesting the same missing txids
when a new block is received in the meantime, causing the top
block to not be the one we need
- send the original fluffy block message block height when sending
a new fluffy block, not the current top height, which might
have been updated since
- avoid sending back the whole block blob when asking for txes,
send only the hash instead
- plus misc cleanup and additional debugging logs
3ff54bdd Check for correct thread before ending batch transaction (Howard Chu)
eaf8470b Must wait for previous batch to finish before starting new one (Howard Chu)
c903c554 Don't cache block height, always get from DB (Howard Chu)
eb1fb601 Tweak default db-sync-mode to fast:async:1 (Howard Chu)
0693cff9 Use batch transactions when syncing (Howard Chu)
If a checksum word is present, language detection would use
just the word prefixes. However, a set of word prefixes may
be found in more than one language, and so the wrong language
may be found first, which could then fail the checksum, since
the check may be done with a different unique prefix length
from the one it was created from.
We now make a checksum test when we we detect a language from
prefixes only, to make sure we have the correct one.
5783dd8c tests: add unit tests for uri parsing (moneromooo-monero)
82ba2108 wallet: add API and RPC to create/parse monero: URIs (moneromooo-monero)
d9001b43 epee: add functions to convert from URL format (ie, %XX values) (moneromooo-monero)
The fee will vary based on the base reward and the current
block size limit:
fee = (R/R0) * (M0/M) * F0
R: base reward
R0: reference base reward (10 monero)
M: block size limit
M0: minimum block size limit (60000)
F0: 0.002 monero
Starts applying at v4
bba6af9 wallet: cold wallet transaction signing (moneromooo-monero)
9872dcb wallet: fix log confusion between bytes and kilobytes (moneromooo-monero)
d9b0bf9 cryptonote_core: make extra field removal more generic (moneromooo-monero)
98f19d4 serialization: add support for serializing std::pair and std::list (moneromooo-monero)
This change adds the ability to create a new unsigned transaction
from a watch only wallet, and save it to a file. This file can
then be moved to another computer/VM where a cold wallet may load
it, sign it, and save it. That cold wallet does not need to have
a blockchain nor daemon. The signed transaction file can then be
moved back to the watch only wallet, which can load it and send
it to the daemon.
Two new simplewallet commands to use it:
sign_transfer (on the cold wallet)
submit_transfer (on the watch only wallet)
The transfer command used on a watch only wallet now writes an
unsigned transaction set in a file called 'unsigned_monero_tx'
instead of submitting the tx to the daemon as a normal wallet does.
The signed tx file is called 'signed_monero_tx'.
Keep the immediate direct deps at the library that depends on them,
declare deps as PUBLIC so that targets that link against that library
get the library's deps as transitive deps.
Break dep cycle between blockchain_db <-> crytonote_core.
No code refactoring, just hide cycle from cmake so that
it doesn't complain (cycles are allowed only between
static libs, not shared libs).
This is in preparation for supproting BUILD_SHARED_LIBS cmake
built-in option for building internal libs as shared.
Since this queries block heights for blocks that may or may not
exist, queries for non existing blocks would throw an exception,
and that would slow down the loop a lot. 7 seconds to go through
a 30 hash list.
Fix this by adding an optional return block height to block_exists
and using this instead. Actual errors will still throw an
exception.
This also cuts down on log exception spam.
This allows the key to be not the same for two outputs sent to
the same address (eg, if you pay yourself, and also get change
back). Also remove the key amounts lists and return parameters
since we don't actually generate random ones, so we don't need
to save them as we can recalculate them when needed if we have
the correct keys.
The whole rct data apart from the MLSAGs is now included in
the signed message, to avoid malleability issues.
Instead of passing the data that's not serialized as extra
parameters to the verification API, the transaction is modified
to fill all that information. This means the transaction can
not be const anymore, but it cleaner in other ways.
Since these are needed at the same time as the output pubkeys,
this is a whole lot faster, and takes less space. Only outputs
of 0 amount store the commitment. When reading other outputs,
a fake commitment is regenerated on the fly. This avoids having
to rewrite the database to add space for fake commitments for
existing outputs.
This code relies on two things:
- LMDB must support fixed size records per key, rather than
per database (ie, all records on key 0 are the same size, all
records for non 0 keys are same size, but records from key 0
and non 0 keys do have different sizes).
- the commitment must be directly after the rest of the data
in outkey and output_data_t.
The mixRing (output keys and commitments) and II fields (key images)
can be reconstructed from vin data.
This saves some modest amount of space in the tx.
It may be suboptimal, but it's a pain to have to rebuild everything
when some of this changes.
Also, no clue why there seems to be two different code paths for
serializing a tx...
This plugs a privacy leak from the wallet to the daemon,
as the daemon could previously see what input is included
as a transaction input, which the daemon hadn't previously
supplied. Now, the wallet requests a particular set of
outputs, including the real one.
This can result in transactions that can't be accepted if
the wallet happens to select too many outputs with non standard
unlock times. The daemon could know this and select another
output, but the wallet is blind to it. It's currently very
unlikely since I don't think anything uses non default
unlock times. The wallet requests more outputs than necessary
so it can use spares if any of the returns outputs are still
locked. If there are not enough spares to reach the desired
mixin, the transaction will fail.
Compilation of bitmonero on Arch with gcc 6.1 results in the following
error:
/home/mwo/bitmonero/tests/unit_tests/hardfork.cpp: In member function ‘virtual void TestDB::set_hard_fork_version(uint64_t, uint8_t)’:
/home/mwo/bitmonero/tests/unit_tests/hardfork.cpp:132:5: error: this ‘if’ clause does not guard... [-Werror=misleading-indentation]
if (versions.size() <= height) versions.resize(height+1); versions[height] = version;
This can be fixed by simply unfolding this line into three lines.
This is a list of existing output amounts along with the number
of outputs of that amount in the blockchain.
The daemon command takes:
- no parameters: all outputs with at least 3 instances
- one parameter: all outputs with at least that many instances
- two parameters: all outputs within that many instances
The default starts at 3 to avoid massive spamming of all dust
outputs in the blockchain, and is the current minimum mixin
requirement.
An optional vector of amounts may be passed, to request
histogram only for those outputs.
7658ac0 blockchain: revert handle_get_objects adding block id on tx not found (moneromooo-monero)
3a0f4d8 berkeleydb: fix delete/free mismatch (moneromooo-monero)
1642be2 minor bugfixes and refactoring (Thomas Winget)
098dcf2 unit_tests: fix mnemonics unit test testing invalid seeds (moneromooo-monero)
Some word triplets, such as "mugged names nail", are not valid
results from any 32 bit value. If used to decode a 32 bit value,
the result will therefore encode to a different word triplet.
Fix this by using random words converted from an actual random
bitstring, ensuring we always get valid triplets.
Also add some more tests, and rename some instances of
"version" and "add" for clarity.
NOTE: the starting height values are sometimes wrong.
I suspect this is due to the hard fork reorg code being
buggy, since they're good when syncing after the fact.
However, they're not actually used by the consensus code,
so I'm ignoring this for now, but this needs debugging.
Using major version would cause older daemons to reject those
blocks as they fail to deserialize blocks with a major version
which is not 1. There is no such restriction on the minor
version, so switching allows older daemons to coexist with
newer ones till the actual fork date, when most will hopefully
have updated already.
Also, for the same reason, we consider a vote for 0 to be a
vote for 1, since older daemons set minor version to 0.
This allows knowing the hard fork a block must obey in order to be
added to the blockchain. The previous semantics would use that new
block's version vote to determine this hard fork, which made it
impossible to use the rules to validate transactions entering the
tx pool (and made it impossible to validate a block before adding
it to the blockchain).
This ensures one can't instanciate a DNSResolver object by
mistake, but uses the singleton. A separate create static
function is added for cases where a new object is explicitely
needed.
Based on tewinget's update.
Make OpenAlias address format independent of existing DNS functions.
Add tests.
Test:
make debug-test
cd build/debug/tests/unit_tests
# test that regular DNS functions work, including IPv4 lookups.
# also test function that converts OpenAlias address format
make && ./unit_tests --gtest_filter=DNSResolver*
# test that OpenAlias addresses like donate@getmonero.org work from
# wallet tools
make && ./unit_tests --gtest_filter=AddressFromURL.Success
DNSSEC is now implemented with the hardcoded key from unbound.
This will need to be not hardcoded in the future, but is okay for now.
Unit tests updated for DNSSEC (as well as for the fact that, contrary to
previous assumption, example.com does not have a static IP address).
Everything except actually *using* BlockchainBDB is wired up, but the db
itself is not yet working. Some error about user mem not large enough.
I think I know what this error means, but I can't determine the cause.
Notes: BerkeleyDB does not allow 0-indexing in its recno type databases,
so block numbers *in the database* will be 1-indexed. Modifications
to indexing have been made as needed.
+toc -doc -drmonero
Fixed the windows path, and improved logging and data
(for graph) logging, fixed some locks and added more checks.
Still there is a locking error,
not added by my patches, but present in master version
(locking of map/list of peers).
commands and options for network limiting
works very well e.g. for 50 KiB/sec up and down
ToS (QoS) flag
peer number limit
TODO some spikes in ingress/download
TODO problems when other up and down limit
added "otshell utils" - simple logging (with colors, text files channels)
Some BlockchainDB unit testing fleshed out (and working), rudimentary
linker flag for lmdb in CMakeLists, but should probably be done
"correctly" at some point (find it on whatever system you're building on
and all that jazz).
update for rebase (warptangent 2015-01-04)
fix conflicts with upstream CMakeLists.txt files
tests/CMakeLists.txt (remove edits from original commit)
b94b8cd Added unit test. Fails for Japanese for some reason. (Oran Juice)
4c8a628 Remove iostream header put in during testing (Oran Juice)
9875f5b Variable unique prefix lengths for seed (Oran Juice)
