
­­­­­­­­­­­­­­­­­­
<!DOCTYPE html>
<html>
3
yb                 @   s  d dl Z d dlZd dlZd dlZd dlZd dlZd dlmZ d dlZddl	m
Z
 ddlmZ ddlmZ ddlmZ ddlmZmZ dd	lmZmZmZmZmZ dd
lmZmZmZmZmZmZmZ ddl m!Z! ddl	m"Z" e# Z$G dd de!Z%G dd de%Z&G dd de%Z'G dd de%Z(G dd de%Z)G dd de%Z*G dd de%Z+G dd de%Z,G dd de%Z-G d d! d!e%Z.G d"d# d#e%Z/G d$d% d%e&Z0G d&d' d'e%Z1G d(d) d)e.Z2G d*d+ d+e!Z3dS ),    N)patch   )NSIndex)Location)IntegrityError)msgpack)Lock
LockFailed)RemoteRepositoryInvalidRPCMethodPathNotAllowedConnectionClosedWithHinthandle_remote_line)
RepositoryLoggedIOMAGICMAX_DATA_SIZE
TAG_DELETETAG_PUT
TAG_COMMIT   )BaseTestCase)Hc               @   sP   e Zd ZdZdZdefddZdd Zdd	 Zefd
dZ	dd Z
dddZdS )RepositoryTestCaseBase    TFc             C   s(   |t kr| j}ttjj| jd||dS )N
repository)	exclusivecreate)UNSPECIFIEDr   r   ospathjointmppath)selfr   r    r$   "/usr/lib64/python3.6/repository.pyopen   s    zRepositoryTestCaseBase.openc             C   s&   t j | _| jdd| _| jj  d S )NT)r   )tempfileZmkdtempr"   r&   r   	__enter__)r#   r$   r$   r%   setUp"   s    
zRepositoryTestCaseBase.setUpc             C   s   | j j  tj| j d S )N)r   closeshutilZrmtreer"   )r#   r$   r$   r%   tearDown'   s    
zRepositoryTestCaseBase.tearDownc             C   s"   | j r| j j  | j|d| _ d S )N)r   )r   r*   r&   )r#   r   r$   r$   r%   reopen+   s    
zRepositoryTestCaseBase.reopenc             C   sx   | j jtdd | j jtdd | j jtdd | j j  | j jtdd | j jtdd | j jtd d S )	Nr   s   foor   s   bar   s   bar2r   s   boo)r   putr   commitdelete)r#   r$   r$   r%   add_keys0   s    
zRepositoryTestCaseBase.add_keysNc       
   
   C   s   |d k	r|d nd}dd t dD }d|d < tdtdtd	i}xX| jjj D ]H\}}x>| jjj|D ],\}}}}	td
||| || |||	f  qdW qLW t  d S )Nz:  c             S   s   i | ]}|t |qS r$   )r   ).0ir$   r$   r%   
<dictcomp>;   s    z4RepositoryTestCaseBase.repo_dump.<locals>.<dictcomp>
   r   r/   delZcommz%s%s H(%d) -> %s[%d..+%d])	ranger   r   r   r   iosegment_iteratoriter_objectsprint)
r#   ZlabelZH_transZ	tag_transsegmentfntagkeyoffsetsizer$   r$   r%   	repo_dump9   s    (z RepositoryTestCaseBase.repo_dump)N)__name__
__module____qualname__Zkey_sizer   r   r&   r)   r,   r-   r2   rE   r$   r$   r$   r%   r      s   	r   c               @   sT   e Zd Zdd Zdd Zdd Zdd Zd	d
 Zdd Zdd Z	dd Z
dd ZdS )RepositoryTestCasec                s   x"t dD ]}jjt|d q
W td jjj d jj  jtj	 fdd jj
  jj  j Pjtj	 fdd x0t dD ]$}|dkrqjjt|d qW W d Q R X d S )Nd   s   SOMEDATA2   c                  s   j j S )N)r   getr$   )key50r#   r$   r%   <lambda>L   s    z*RepositoryTestCase.test1.<locals>.<lambda>c                  s
   j  S )N)rL   r$   )rM   repository2r$   r%   rN   P   s    )r:   r   r/   r   assert_equalrL   r1   assert_raisesr   ObjectNotFoundr0   r*   r&   )r#   xr$   )rM   rO   r#   r%   test1F   s    


zRepositoryTestCase.test1c             C   sv   | j jtdd | j jtdd | j j  | j jtd | j jtdd | j j  | j| j jtdd dS )z.Test multiple sequential transactions
        r   s   foor   s   barN)r   r/   r   r0   r1   rP   rL   )r#   r$   r$   r%   test2V   s    

zRepositoryTestCase.test2c                s    j jtdd  j j jtdd  j jtdd  j j jtdd  j jtdd  j j jtdd  j jtd  jtj fdd dS )zTest cache consistency
        r   s   foos   foo2s   barc                  s    j jtdS )Nr   )r   rL   r   r$   )r#   r$   r%   rN   k   s    z5RepositoryTestCase.test_consistency.<locals>.<lambda>N)	r   r/   r   rP   rL   r1   rQ   r   rR   )r#   r$   )r#   r%   test_consistencya   s    z#RepositoryTestCase.test_consistencyc             C   s   | j jtdd | j| j jtdd | j j  | j jtdd | j| j jtdd | j j  | j| j jtdd dS )z Test cache consistency2
        r   s   foos   foo2N)r   r/   r   rP   rL   r0   rollback)r#   r$   r$   r%   test_consistency2m   s    

z$RepositoryTestCase.test_consistency2c             C   sJ   | j jtdd | j jtdd | j j  | j| j jtdd dS )z Test cache consistency2
        r   s   foos   foo2N)r   r/   r   r0   rP   rL   )r#   r$   r$   r%   "test_overwrite_in_same_transactionx   s    
z5RepositoryTestCase.test_overwrite_in_same_transactionc             C   s   | j jtdd | j j  | j j  | j | _ | j " | j jtdd | j j  W d Q R X | j | _ | j   | j jtd | j j  W d Q R X d S )Nr   s   foos   bar)r   r/   r   r0   r*   r&   r1   )r#   r$   r$   r%   test_single_kind_transactions   s    



z0RepositoryTestCase.test_single_kind_transactionsc             C   s   x"t dD ]}| jjt|d q
W | jj  | jj }| jt|d | jjdd}| jt|d | j||d d  | jj|d d}| jt|d | j||dd   | jt| jjddd d S )NrJ   s   SOMEDATArK   )limitr   )markerr9   )r:   r   r/   r   r0   listrP   len)r#   rS   all
first_halfsecond_halfr$   r$   r%   	test_list   s    

zRepositoryTestCase.test_listc             C   sn   x"t dD ]}| jjt|d q
W | jj  | jj }| jjdd}| jj|d d}xt dD ]}qbW d S )NrJ   s   SOMEDATArK   )r[   r   )r\   r9   )r:   r   r/   r   r0   scan)r#   rS   r_   r`   ra   r$   r$   r%   	test_scan   s    

zRepositoryTestCase.test_scanc                sL   dt   jjtd  jjjtd  jt fdd d S )N   xr   c                  s   j jtd d S )Nr   re   )r   r/   r   r$   )max_datar#   r$   r%   rN      s    z7RepositoryTestCase.test_max_data_size.<locals>.<lambda>)r   r   r/   r   rP   rL   rQ   r   )r#   r$   )rf   r#   r%   test_max_data_size   s
    z%RepositoryTestCase.test_max_data_sizeN)rF   rG   rH   rT   rU   rV   rX   rY   rZ   rb   rd   rg   r$   r$   r$   r%   rI   D   s   rI   c               @   s4   e Zd Zdd Zdd Zdd Zdd Zd	d
 ZdS )LocalRepositoryTestCasec             C   s   | j jd d S )Nr   )r   _rebuild_sparse)r#   r$   r$   r%   _assert_sparse   s    z&LocalRepositoryTestCase._assert_sparsec             C   sL   | j jtdd | j jtdd | j j  | j jtdd | j  d S )Nr   s   foor   s	   123456789s   bar)r   r/   r   r0   rj   )r#   r$   r$   r%   test_sparse1   s
    
z$LocalRepositoryTestCase.test_sparse1c             C   sJ   | j jtdd | j jtdd | j j  | j jtd | j  d S )Nr   s   foor   s	   123456789)r   r/   r   r0   r1   rj   )r#   r$   r$   r%   test_sparse2   s
    
z$LocalRepositoryTestCase.test_sparse2c             C   sJ   | j jtdd | j jtd | j jjj  | j jd | j j  d S )Nr   s   1245)	r   r/   r   r1   r;   Z	_write_fdsyncri   r0   )r#   r$   r$   r%   test_sparse_delete   s    
z*LocalRepositoryTestCase.test_sparse_deletec             C   s   | j jtdd | j j  | j jj }t| j jj|d d}|jt	d  W d Q R X | j j
  | j | _ | j " | j jtdd | j j  W d Q R X d S )Nr   s   foor   wbs   crapcrapcraps   bar)r   r/   r   r0   r;   get_latest_segmentr&   Zsegment_filenamewriter   r*   )r#   last_segmentfr$   r$   r%   test_uncommitted_garbage   s    


z0LocalRepositoryTestCase.test_uncommitted_garbageN)rF   rG   rH   rj   rk   rl   rn   rt   r$   r$   r$   r%   rh      s
   rh   c               @   s\   e Zd Zdd Zdd Zdd Zdd Zd	d
 Zdd Zdd Z	dd Z
dd Zdd ZdS )RepositoryCommitTestCasec          
   C   s   | j   x8tj| jjD ]&}|jdrtjtjj| jj| qW | j  | j* | j	t
| jd | j	| jj d W d Q R X d S )Nzindex.r.   T)r2   r   listdirr   r    
startswithunlinkr!   r-   rP   r^   check)r#   namer$   r$   r%   test_replay_of_missing_index   s    
z5RepositoryCommitTestCase.test_replay_of_missing_indexc             C   sv   | j   d | j_y| jj  W n tk
r2   Y nX | j  | j* | jt| jd | j| jj d W d Q R X d S )Nr.   T)	r2   r   compact_segmentsr0   	TypeErrorr-   rP   r^   ry   )r#   r$   r$   r%   "test_crash_before_compact_segments   s    z;RepositoryCommitTestCase.test_crash_before_compact_segmentsc             C   sv   | j   d | j_y| jj  W n tk
r2   Y nX | j  | j* | jt| jd | j| jj d W d Q R X d S )Nr.   T)	r2   r   Zwrite_indexr0   r}   r-   rP   r^   ry   )r#   r$   r$   r%   test_crash_before_write_index  s    z6RepositoryCommitTestCase.test_crash_before_write_indexc                s    j   x8tj jjD ]&}|jdrtjtjj jj| qW tj	t
dtd@} jd d  j"  jt fdd |j  W d Q R X W d Q R X d S )Nzindex.upgrade)side_effect)r   c                  s
   t  jS )N)r^   r   r$   )r#   r$   r%   rN     s    zGRepositoryCommitTestCase.test_replay_lock_upgrade_old.<locals>.<lambda>)r2   r   rv   r   r    rw   rx   r!   r   objectr   r	   r-   rQ   assert_called_once_with)r#   rz   r   r$   )r#   r%   test_replay_lock_upgrade_old  s    
z5RepositoryCommitTestCase.test_replay_lock_upgrade_oldc                s    j   x8tj jjD ]&}|jdrtjtjj jj| qW tj	t
dtd@} jdd  j"  jt fdd |j  W d Q R X W d Q R X d S )Nzindex.r   )r   F)r   c                  s
   t  jS )N)r^   r   r$   )r#   r$   r%   rN   +  s    zCRepositoryCommitTestCase.test_replay_lock_upgrade.<locals>.<lambda>)r2   r   rv   r   r    rw   rx   r!   r   r   r   r	   r-   rQ   r   )r#   rz   r   r$   )r#   r%   test_replay_lock_upgrade!  s    
z1RepositoryCommitTestCase.test_replay_lock_upgradec             C   s   | j   d | jj_y| jj  W n tk
r4   Y nX | j  | j< | jt| jd | j| jj	 d | jt| jd W d Q R X d S )Nr.   T)
r2   r   r;   delete_segmentr0   r}   r-   rP   r^   ry   )r#   r$   r$   r%   -test_crash_before_deleting_compacted_segments.  s    
zFRepositoryCommitTestCase.test_crash_before_deleting_compacted_segmentsc          	   C   s:   | j jtdtj | j  | j  | j j}W d Q R X d S )Nr   )r   r/   r   r   ZCOMMITr-   r;   )r#   r;   r$   r$   r%   test_ignores_commit_tag_in_data;  s
    z8RepositoryCommitTestCase.test_ignores_commit_tag_in_datac       	      C   s  | j jtdd | j jtdd | j j  | jd | j jtd | j j  | jd | j jj d }d}x.| j jj|D ]\}}}}|t	kr|d7 }qW | j jtdd	 | j j  | jd
 x8| j jj
 D ](\}}x| j jj|D ]\}}}}qW qW d S )Nr      1r      2zp1 p2 cczd1 ccr   r.      3zp3 cc)r   r/   r   r0   rE   r1   r;   rp   r=   r   r<   )	r#   rr   Znum_deletesrA   rB   rC   rD   r?   _r$   r$   r%   test_moved_deletes_are_trackedB  s&    





z7RepositoryCommitTestCase.test_moved_deletes_are_trackedc             C   s   | j jj}| j jtdd | }| j j  | j jtd | j jtdd | }| j j|= | j j|= | j j  | j jtd | j j  tj	tj
j| j j
d|   d S )Nr   r   r   zindex.%d)r   r;   rp   r/   r   r0   r1   compactr   rx   r    r!   )r#   rp   Zput_segmentr   r$   r$   r%   #test_shadowed_entries_are_preserved_  s    





z<RepositoryCommitTestCase.test_shadowed_entries_are_preservedc             C   s   | j jtdd | j jtd | j j  | jd | j jtdd | j jtd | jd | j j  | jd | j jtdd d S )Nr   r   zp1 d1 cczp1 d1rr   )r   r/   r   r1   r0   rE   rW   )r#   r$   r$   r%   test_shadow_index_rollback  s    




z3RepositoryCommitTestCase.test_shadow_index_rollbackN)rF   rG   rH   r{   r~   r   r   r   r   r   r   r   r   r$   r$   r$   r%   ru      s   
"ru   c               @   s&   e Zd Zd	ddZdd Zdd ZdS )
RepositoryAppendOnlyTestCaseFc             C   s   t tjj| jdd|ddS )Nr   T)r   r   append_only)r   r   r    r!   r"   )r#   r   r$   r$   r%   r&     s    z!RepositoryAppendOnlyTestCase.openc          	   C   s$   | j t | jj  W d Q R X d S )N)rQ   
ValueErrorr   Zdestroy)r#   r$   r$   r%   test_destroy_append_only  s    z5RepositoryAppendOnlyTestCase.test_destroy_append_onlyc                st    fdd} j jtdd  j j  d j _ j jtdd  j j  d j _ j jtdd  j j  d S )Nc                  s   t t jjj S )N)r^   r]   r   r;   r<   r$   )r#   r$   r%   segments_in_repository  s    zMRepositoryAppendOnlyTestCase.test_append_only.<locals>.segments_in_repositoryr   s   fooFT)r   r/   r   r0   r   )r#   r   r$   )r#   r%   test_append_only  s    


z-RepositoryAppendOnlyTestCase.test_append_onlyN)F)rF   rG   rH   r&   r   r   r$   r$   r$   r%   r     s   
r   c               @   s   e Zd Zdd Zdd ZdS )RepositoryFreeSpaceTestCasec             C   sx   | j   | jjjddd | jjd | j  | j: | jjtdd tj	t
j | jj  W d Q R X W d Q R X d S )Nr   additional_free_spaceZ1000Ts   shortcut to save_configr   s   foobar)r2   r   configsetZsave_keyr-   r/   r   pytestraisesr   InsufficientFreeSpaceErrorr0   )r#   r$   r$   r%   test_additional_free_space  s    z6RepositoryFreeSpaceTestCase.test_additional_free_spacec          	   C   s,   d| j _tjtj | j  W d Q R X d S )Ng@xD)r   r   r   r   r   r   r2   )r#   r$   r$   r%   test_create_free_space  s    z2RepositoryFreeSpaceTestCase.test_create_free_spaceN)rF   rG   rH   r   r   r$   r$   r$   r%   r     s   r   c               @   s   e Zd Zdd Zdd ZdS )QuotaTestCasec          
   C   s   | j jtdtd | j jtdtd | j jtd | j j  | j  | j 2 | j jtdd | j jtd | j j  W d Q R X d S )Nr   i  r   i.  r.       )r   r/   r   bytesr1   r0   r-   )r#   r$   r$   r%   test_tracking  s    

zQuotaTestCase.test_trackingc             C   s   d| j _| j jtdd | j j  tjtj | j jtdd W d Q R X tjtj | j j  W d Q R X | j	  | j * d| j _| j jtdd | j j  W d Q R X d S )NrK   r   r   r   rJ   )
r   storage_quotar/   r   r0   r   r   r   ZStorageQuotaExceededr-   )r#   r$   r$   r%   test_exceed_quota  s    

zQuotaTestCase.test_exceed_quotaN)rF   rG   rH   r   r   r$   r$   r$   r%   r     s   r   c               @   s,   e Zd Zdd Zdd Zdd Zdd Zd	S )
NonceReservationc             C   sB   | j dd tjt" | j | jj  W d Q R X W d Q R X d S )NF)r   )r-   r   r   AssertionErrorr   Zget_free_nonce)r#   r$   r$   r%   test_get_free_nonce_asserts  s    z,NonceReservation.test_get_free_nonce_assertsc             C   sr   | j b ttjj| j jdd}|jd W d Q R X ttjj| j jdd}|jd W d Q R X W d Q R X d S )NnoncewZ0000000000000000Z5000000000000000)r   r&   r   r    r!   rq   )r#   fdr$   r$   r%   test_get_free_nonce  s    z$NonceReservation.test_get_free_noncec             C   sF   | j dd tjt& | j | jjdd W d Q R X W d Q R X d S )NF)r   i      )r-   r   r   r   r   commit_nonce_reservation)r#   r$   r$   r%   %test_commit_nonce_reservation_asserts  s    z6NonceReservation.test_commit_nonce_reservation_assertsc          %   C   s   | j  tjt | j jdd W d Q R X | j jdd  ttjj| j jdd}W d Q R X tjt | j jdd W d Q R X | j jdd ttjj| j jdd}W d Q R X W d Q R X d S )Ni      r   r   i   )	r   r   r   	Exceptionr   r&   r   r    r!   )r#   r   r$   r$   r%   test_commit_nonce_reservation  s    
z.NonceReservation.test_commit_nonce_reservationN)rF   rG   rH   r   r   r   r   r$   r$   r$   r%   r     s   r   c                   s   e Zd Z fddZdd Zdd Zdd Zd	d
 Zdd Zdd Z	dd Z
dd Zdd Zdd Zdd Zdd Zdd Zdd Zdd  Z  ZS )!%RepositoryAuxiliaryCorruptionTestCasec                s4   t  j  | jjtdd | jj  | jj  d S )Nr   s   foo)superr)   r   r/   r   r0   r*   )r#   )	__class__r$   r%   r)     s    

z+RepositoryAuxiliaryCorruptionTestCase.setUpc          
   C   s2   | j " | j jtdd | j j  W d Q R X d S )Nr   s   fox)r   r/   r   r0   )r#   r$   r$   r%   	do_commit#  s    z/RepositoryAuxiliaryCorruptionTestCase.do_commitc          	   C   s:   t tjj| jjdd}|jd W d Q R X | j  d S )Nzhints.1Zabs	   123456789)r&   r   r    r!   r   rq   r   )r#   r   r$   r$   r%   test_corrupted_hints(  s    z:RepositoryAuxiliaryCorruptionTestCase.test_corrupted_hintsc             C   s$   t jt jj| jjd | j  d S )Nzhints.1)r   rx   r    r!   r   r   )r#   r$   r$   r%   test_deleted_hints-  s    z8RepositoryAuxiliaryCorruptionTestCase.test_deleted_hintsc             C   s$   t jt jj| jjd | j  d S )Nzindex.1)r   rx   r    r!   r   r   )r#   r$   r$   r%   test_deleted_index1  s    z8RepositoryAuxiliaryCorruptionTestCase.test_deleted_indexc          	   C   sH   t jj| jjd}t j| t j| | jt | j  W d Q R X d S )Nzhints.1)	r   r    r!   r   rx   mkdirrQ   OSErrorr   )r#   hintsr$   r$   r%   test_unreadable_hints5  s
    

z;RepositoryAuxiliaryCorruptionTestCase.test_unreadable_hintsc          	   C   s:   t tjj| jjdd}|jd W d Q R X | j  d S )Nzindex.1ro   s	   123456789)r&   r   r    r!   r   rq   r   )r#   r   r$   r$   r%   
test_index<  s    z0RepositoryAuxiliaryCorruptionTestCase.test_indexc             C   sD   t tjj| jjdd}|jd W d Q R X | j W d Q R X d S )Nzindex.1ro   s	   123456789)r&   r   r    r!   r   rq   )r#   r   r$   r$   r%   test_index_outside_transactionA  s    zDRepositoryAuxiliaryCorruptionTestCase.test_index_outside_transactionc          
   C   st   t jj| jjd}t|dN}|j }tjtdddA j	dd}|j
td|}|jd |j| W d Q R X d S )Nzindex.1zr+br   littler   r   )r   r    r!   r   r&   readint
from_bytesr   to_bytesreplaceseekrq   )r#   Z
index_pathr   Z
index_dataZcorrupted_keyZcorrupted_index_datar$   r$   r%   _corrupt_indexG  s    
z4RepositoryAuxiliaryCorruptionTestCase._corrupt_indexc          	   C   s   | j   | j W d Q R X d S )N)r   r   )r#   r$   r$   r%   test_index_corruptedU  s    z:RepositoryAuxiliaryCorruptionTestCase.test_index_corruptedc             C   sb   | j   tjj| jjd}tj| | j. tjtj	 | jj
td W d Q R X W d Q R X d S )Nzintegrity.1r   )r   r   r    r!   r   rx   r   r   r   rR   rL   r   )r#   integrity_pathr$   r$   r%   &test_index_corrupted_without_integrity_  s    
zLRepositoryAuxiliaryCorruptionTestCase.test_index_corrupted_without_integrityc          	   C   sH   t jj| jjd}t j| t j| | jt | j  W d Q R X d S )Nzindex.1)	r   r    r!   r   rx   r   rQ   r   r   )r#   indexr$   r$   r%   test_unreadable_indexk  s
    

z;RepositoryAuxiliaryCorruptionTestCase.test_unreadable_indexc             C   sV   t jj| jjd}t|d}tjddi| |j  W d Q R X | j W d Q R X d S )Nzintegrity.1zr+bs   versiong@)r   r    r!   r   r&   r   packtruncate)r#   r   r   r$   r$   r%   test_unknown_integrity_versionr  s    zDRepositoryAuxiliaryCorruptionTestCase.test_unknown_integrity_versionc             C   s   | j X d| j _| j jtdd | j jtdd | j j  | j jtdd | j j  W d Q R X tjj| j jd}t|d:}t	j
|}|jd	 d	|d
 d< t	j|| |j  W d Q R X d S )NTr   s   barr   s   bazs   bazzzhints.5zr+br   s   segments)r   r   r/   r   r0   r   r    r!   r&   r   unpackr   r   r   )r#   Z
hints_pathr   r   r$   r$   r%   _subtly_corrupted_hints_setup  s    


zCRepositoryAuxiliaryCorruptionTestCase._subtly_corrupted_hints_setupc          
   C   sB   | j   | j* d| j_| jjtdd | jj  W d Q R X d S )NFr.   s   1234)r   r   r   r/   r   r0   )r#   r$   r$   r%   test_subtly_corrupted_hints  s    
zARepositoryAuxiliaryCorruptionTestCase.test_subtly_corrupted_hintsc             C   st   | j   tjj| jjd}tj| | j@ d| j_| jjtdd t	j
t}| jj  W d Q R X W d Q R X d S )Nzintegrity.5Fr.   s   1234)r   r   r    r!   r   rx   r   r/   r   r   r   r   r0   )r#   r   exc_infor$   r$   r%   -test_subtly_corrupted_hints_without_integrity  s    
zSRepositoryAuxiliaryCorruptionTestCase.test_subtly_corrupted_hints_without_integrity)rF   rG   rH   r)   r   r   r   r   r   r   r   r   r   r   r   r   r   r   r   __classcell__r$   r$   )r   r%   r     s    
r   c               @   s   e Zd Zdd Zd*ddZdd Zd	d
 Zdd Zdd Zdd Z	dd Z
dd Zdd Zdd Zdd Zdd Zdd Zdd  Zd!d" Zd#d$ Zd%d& Zd'd( Zd)S )+RepositoryCheckTestCasec             C   s    dd t jt jj| jdD S )Nc             S   s   g | ]}|j d r|qS )zindex.)rw   )r4   rz   r$   r$   r%   
<listcomp>  s    z8RepositoryCheckTestCase.list_indices.<locals>.<listcomp>r   )r   rv   r    r!   r"   )r#   r$   r$   r%   list_indices  s    z$RepositoryCheckTestCase.list_indicesFTc             C   sD   | j | jj|d| | j dd tjtjj| jdD g d d S )N)repairc             S   s   g | ]}d |kr|qS )Ztmpr$   )r4   rz   r$   r$   r%   r     s    z1RepositoryCheckTestCase.check.<locals>.<listcomp>r   zFound tmp files)rP   r   ry   r   rv   r    r!   r"   )r#   r   statusr$   r$   r%   ry     s    zRepositoryCheckTestCase.checkc             G   s"   x|D ]}| j jt| qW d S )N)r   rL   r   )r#   idsid_r$   r$   r%   get_objects  s    
z#RepositoryCheckTestCase.get_objectsc             C   s<   x6|D ].}x|D ]}| j jt|d qW | j j  qW d S )Ns   data)r   r/   r   r0   )r#   segmentsr   r   r$   r$   r%   add_objects  s    

z#RepositoryCheckTestCase.add_objectsc             C   s,   t dd tjtjj| jdddD d S )Nc             s   s   | ]}|j  rt|V  qd S )N)isdigitr   )r4   nr$   r$   r%   	<genexpr>  s    z3RepositoryCheckTestCase.get_head.<locals>.<genexpr>r   data0r   r9   )sortedr   rv   r    r!   r"   )r#   r$   r$   r%   get_head  s    z RepositoryCheckTestCase.get_headc             C   s"   t jtjj| jddj| j S )Nr   zindex.{})r   r   r   r    r!   r"   formatr   )r#   r$   r$   r%   
open_index  s    z"RepositoryCheckTestCase.open_indexc          	   C   s\   | j  }|t| \}}ttjj| jdddt|d}|j| |j	d W d Q R X d S )Nr   r   r   zr+bs   BOOM)
r   r   r&   r   r    r!   r"   strr   rq   )r#   r   idxr?   rC   r   r$   r$   r%   corrupt_object  s
    "
z&RepositoryCheckTestCase.corrupt_objectc             C   s$   t jt jj| jdddt| d S )Nr   r   r   )r   rx   r    r!   r"   r   )r#   r?   r$   r$   r%   r     s    z&RepositoryCheckTestCase.delete_segmentc             C   s&   t jt jj| jddj| j  d S )Nr   zindex.{})r   rx   r    r!   r"   r   r   )r#   r$   r$   r%   delete_index  s    z$RepositoryCheckTestCase.delete_indexc             C   s6   t jt jj| jddj| j t jj| jd| d S )Nr   zindex.{})r   renamer    r!   r"   r   r   )r#   new_namer$   r$   r%   rename_index  s    z$RepositoryCheckTestCase.rename_indexc             C   s   t dd | jj D S )Nc             s   s   | ]}t |V  qd S )N)r   )r4   rB   r$   r$   r%   r     s    z7RepositoryCheckTestCase.list_objects.<locals>.<genexpr>)r   r   r]   )r#   r$   r$   r%   list_objects  s    z$RepositoryCheckTestCase.list_objectsc                s    j dddgddgdgg  jtddddddg j   jdd  jd  jt fd	d
  jj	   jdd  jdd  jddd  j
d  jdd  jtdddddg j  d S )Nr   r   r.            T)r   c                  s
    j dS )Nr   )r   r$   )r#   r$   r%   rN     s    zGRepositoryCheckTestCase.test_repair_corrupted_segment.<locals>.<lambda>F)r   r   )r   rP   r   r   ry   r   rQ   r   r   rW   r   )r#   r$   )r#   r%   test_repair_corrupted_segment  s     


z5RepositoryCheckTestCase.test_repair_corrupted_segmentc             C   s   | j dddgdddgg | jtddddddg| j  | jdd | jd | jj  | jddd	 | jtdddg| j  d S )
Nr   r   r.   r   r   r   T)r   )r   r   )r   rP   r   r   ry   r   r   rW   )r#   r$   r$   r%   test_repair_missing_segment  s     

z3RepositoryCheckTestCase.test_repair_missing_segmentc                sX    j dddgdddgg  jd  jtj fdd  jtdddg j  d S )	Nr   r   r.   r   r   r   c                  s
    j dS )Nr   )r   r$   )r#   r$   r%   rN     s    zLRepositoryCheckTestCase.test_repair_missing_commit_segment.<locals>.<lambda>)r   r   rQ   r   rR   rP   r   r   )r#   r$   )r#   r%   "test_repair_missing_commit_segment  s    
z:RepositoryCheckTestCase.test_repair_missing_commit_segmentc                s    j dddgdddgg ttjj jddd	d
d}|jdtj |jd W d Q R X  j	t
j fdd  jdd  jd  jtdddg j  d S )Nr   r   r.   r   r   r   r   r   r   3zr+b   Xc                  s
    j dS )Nr   )r   r$   )r#   r$   r%   rN     s    zNRepositoryCheckTestCase.test_repair_corrupted_commit_segment.<locals>.<lambda>T)r   r9   )r   r&   r   r    r!   r"   r   SEEK_ENDrq   rQ   r   rR   ry   r   rP   r   r   )r#   r   r$   )r#   r%   $test_repair_corrupted_commit_segment  s    
z<RepositoryCheckTestCase.test_repair_corrupted_commit_segmentc                s    j dddgg ttjj jddddd}|jdtj |jd	 W d Q R X  j	t
j fd
d  jdd  jdd  j j dg  jddd  j j dg  jdd  jd  jtdddg j  d S )Nr   r   r.   r   r   r   1zr+br   c                  s
    j dS )Nr   )r   r$   )r#   r$   r%   rN     s    z@RepositoryCheckTestCase.test_repair_no_commits.<locals>.<lambda>F)r   zindex.1T)r   r   zindex.3r9   )r   r&   r   r    r!   r"   r   r   rq   rQ   r   CheckNeededry   rP   r   r   r   r   )r#   r   r$   )r#   r%   test_repair_no_commits  s    
z.RepositoryCheckTestCase.test_repair_no_commitsc             C   s\   | j dddgdddgg | j  | jdd | jd | jtddddddg| j  d S )	Nr   r   r.   r   r   r   T)r   )r   r   ry   r   rP   r   r   )r#   r$   r$   r%   test_repair_missing_index  s
    
z1RepositoryCheckTestCase.test_repair_missing_indexc             C   s   | j dddgdddgg | j| j dg | jd | jd	d
 | j| j dg | jd | jtddddddg| j  d S )Nr   r   r.   r   r   r   zindex.3z	index.100T)r   )r   rP   r   r   ry   r   r   r   )r#   r$   r$   r%   test_repair_index_too_new  s    

z1RepositoryCheckTestCase.test_repair_index_too_newc             C   s   | j jtdd | j jtdd tjtd}| j j  |j  W d Q R X | j  | j * | j	dd | j
| j jtdd W d Q R X d S )Nr   s   datas   data2r|   T)r   )r   r/   r   r   r   r   r0   r   r-   ry   rP   rL   )r#   r   r$   r$   r%   test_crash_before_compact%  s    
z1RepositoryCheckTestCase.test_crash_before_compactN)FT)rF   rG   rH   r   ry   r   r   r   r   r   r   r   r   r   r   r   r   r   r   r   r   r  r$   r$   r$   r%   r     s&   
	
	r   c               @   s   e Zd Zdd ZdS )RepositoryHintsTestCasec             C   s   | j jtdd | j| j ji  | j jtd | j j  | j jtdd | j j  | jtd| j j | jd| j j | jd| j j	 d S )Nr   s   datar   )
r   r/   r   rP   Zshadow_indexr1   r0   Zassert_not_inr   r   )r#   r$   r$   r%   test_hints_behaviour4  s    

z,RepositoryHintsTestCase.test_hints_behaviourN)rF   rG   rH   r  r$   r$   r$   r%   r  2  s   r  c               @   sB   e Zd ZdZdddZdd Zdd Zd	d
 Zdd Zdd Z	dS )RemoteRepositoryTestCaseNFc             C   s"   t tdtjj| jd d|dS )Nz__testsuite__:r   T)r   r   )r
   r   r   r    r!   r"   )r#   r   r$   r$   r%   r&   H  s    zRemoteRepositoryTestCase.openc             C   s   G dd d}| S )Nc               @   s$   e Zd ZdZdZg ZdZdd ZdS )z9RemoteRepositoryTestCase._get_mock_args.<locals>.MockArgsZborg?   Nc             S   s
   t | |S )N)hasattr)r#   itemr$   r$   r%   __contains__S  s    zFRemoteRepositoryTestCase._get_mock_args.<locals>.MockArgs.__contains__)rF   rG   rH   remote_pathumaskdebug_topicsrshr  r$   r$   r$   r%   MockArgsM  s
   r  r$   )r#   r  r$   r$   r%   _get_mock_argsL  s    
z'RemoteRepositoryTestCase._get_mock_argsc                s    j t fdd d S )Nc                  s    j jdi S )N__init__)r   callr$   )r#   r$   r%   rN   Z  s    z;RemoteRepositoryTestCase.test_invalid_rpc.<locals>.<lambda>)rQ   r   )r#   r$   )r#   r%   test_invalid_rpcY  s    z)RemoteRepositoryTestCase.test_invalid_rpcc          r   C   s  d}y| j jdddi W n$ tjk
r> } zW Y d d }~X nX y| j jdddi W n$ tjk
rz } zW Y d d }~X nX y| j jdddi W n$ tjk
r } zW Y d d }~X nX y| j jdddi W n" tk
r } zW Y d d }~X nX y| j jdddi W n$ tk
r, } zW Y d d }~X nX y| j jddd	i W n& tjk
rj } zW Y d d }~X nX y| j jddd
i W n$ t	k
r } zW Y d d }~X nX y| j jdddi W n& t
jk
r } zW Y d d }~X nX d S )Nztest stringZinject_exceptionZkindDoesNotExistAlreadyExistsr   r   r   rR   r   Zdivide)r   r  r   r  r  r   r   r   rR   r   r
   ZRPCError)r#   s1er$   r$   r%   test_rpc_exception_transport\  sB    z5RemoteRepositoryTestCase.test_rpc_exception_transportc             C   s   | j  }|| j_dtjd< d S )Nz	ssh --fooZBORG_RSH)r  r   _argsr   environ)r#   argsr$   r$   r%   test_ssh_cmd  s    
z%RemoteRepositoryTestCase.test_ssh_cmdc             C   sN   | j  }tj jtj d|_ddg|_| j  }d|_d|_d|_|| j	_
d S )Nzborg-0.28.2Zsomething_client_sideZrepository_compactionr   iz
ssh -i foo)r  logging	getLoggersetLevelINFOr	  r  r   r  r   r  )r#   r  r$   r$   r%   test_borg_cmd  s    
z&RemoteRepositoryTestCase.test_borg_cmd)F)
rF   rG   rH   r   r&   r  r  r  r  r  r$   r$   r$   r%   r  E  s   
6
r  c               @   s   e Zd ZdddZdd ZdS )RemoteLegacyFreeFc             C   s<   t jtdd$ ttdtjj| jd d|dS Q R X d S )NZ
dictFormatTz__testsuite__:r   )r   r   )r   r   r
   r   r   r    r!   r"   )r#   r   r$   r$   r%   r&     s    zRemoteLegacyFree.openc             C   s   | j jtdd | j j  | j j  | j | _ | j " | j jtdd | j j  W d Q R X | j | _ | j   | j jtd | j j  W d Q R X d S )Nr   s   foos   bar)r   r/   r   r0   r*   r&   r1   )r#   r$   r$   r%   test_legacy_free  s    



z!RemoteLegacyFree.test_legacy_freeN)F)rF   rG   rH   r&   r!  r$   r$   r$   r%   r     s   
r   c               @   s   e Zd ZdddZdd ZdS )RemoteRepositoryCheckTestCaseFc             C   s"   t tdtjj| jd d|dS )Nz__testsuite__:r   T)r   r   )r
   r   r   r    r!   r"   )r#   r   r$   r$   r%   r&     s    z"RemoteRepositoryCheckTestCase.openc             C   s   d S )Nr$   )r#   r$   r$   r%   r    s    z7RemoteRepositoryCheckTestCase.test_crash_before_compactN)F)rF   rG   rH   r&   r  r$   r$   r$   r%   r"    s   
r"  c               @   sL   e Zd Zdd Zdd Zdd Zdd Zd	d
 Zdd Zdd Z	dd Z
dS )RemoteLoggerTestCasec             C   s|   t j | _tj| j| _| jgtj jd d < g tjdjd d < g tjdjd d < tj	j
  tj	| _t j  | _	t_	d S )Nzborg.repositoryzborg.repository.foo)r;   StringIOstreamr  StreamHandlerhandlerr  handlerssysstderrflush
old_stderr)r#   r$   r$   r%   r)     s    

zRemoteLoggerTestCase.setUpc             C   s   | j t_d S )N)r,  r)  r*  )r#   r$   r$   r%   r,     s    zRemoteLoggerTestCase.tearDownc             C   s0   t d | j| jj d | j| jj d d S )Nzunstructured stderr message
r3   z$Remote: unstructured stderr message
)r   rP   r%  getvaluer*  )r#   r$   r$   r%   test_stderr_messages  s    z)RemoteLoggerTestCase.test_stderr_messagesc             C   s0   t d | j| jj d | j| jj d d S )Nz%unstructured stderr progress messager3   z-Remote: unstructured stderr progress message)r   rP   r%  r-  r*  )r#   r$   r$   r%   test_stderr_progress_messages  s    z2RemoteLoggerTestCase.test_stderr_progress_messagesc             C   sN   | j jtj tj jtj td | j| jj d | j| j	j d d S )Nz,$LOG INFO Remote: borg < 1.1 format message
z"Remote: borg < 1.1 format message
r3   )
r'  r  r  DEBUGr  r   rP   r%  r-  r*  )r#   r$   r$   r%   test_pre11_format_messages  s
    z/RemoteLoggerTestCase.test_pre11_format_messagesc             C   sN   | j jtj tj jtj td | j| jj d | j| j	j d d S )Nz=$LOG INFO borg.repository Remote: borg >= 1.1 format message
z#Remote: borg >= 1.1 format message
r3   )
r'  r  r  r0  r  r   rP   r%  r-  r*  )r#   r$   r$   r%   test_post11_format_messages  s
    z0RemoteLoggerTestCase.test_post11_format_messagesc             C   sN   | j jtj tj jtj td | j| jj d | j| j	j d d S )Nz:$LOG INFO borg.repository Remote: new format info message
r3   )
r'  r  r  WARNINGr  r   rP   r%  r-  r*  )r#   r$   r$   r%   test_remote_messages_screened  s
    z2RemoteLoggerTestCase.test_remote_messages_screenedc             C   s   t jdjt j t jdjt j | jjt j t j jt j tj }t j|}|jt j |gt jdj	d d < tj }t j|}|jt j |gt jdj	d d < t
d | j|j d | j|j d | j| jj d | j| jj d d S )Nzborg.repositoryzborg.repository.fooz;$LOG INFO borg.repository Remote: new format child message
r3   z!Remote: new format child message
)r  r  r  r  r'  r3  r;   r$  r&  r(  r   rP   r-  r%  r*  )r#   Zchild_streamZchild_handlerZ
foo_streamZfoo_handlerr$   r$   r%    test_info_to_correct_local_child  s"    

z5RemoteLoggerTestCase.test_info_to_correct_local_childN)rF   rG   rH   r)   r,   r.  r/  r1  r2  r4  r5  r$   r$   r$   r%   r#    s   	r#  )4r;   r  r   r+   r)  r'   Zunittest.mockr   r   Z	hashindexr   Zhelpersr   r   r   Zlockingr   r	   Zremoter
   r   r   r   r   r   r   r   r   r   r   r   r   r3   r   r   r   r   r   rI   rh   ru   r   r   r   r   r   r   r  r  r   r"  r#  r$   r$   r$   r%   <module>   sF   $+s9 &**  n