shell bypass 403
a
'�Dg�l � @ s`
d dl Z d dlZd dlZd dlZd dlZd dlZd dlZd dlZd dl Z d dl
Z
d dl Z d dl Z d dl
Z d dlZd dlZd dlZd dlZd dlZd dlZd dlZd dlZd dlZd dlZd dlZd dlZd dlmZ d dl m Z m Z d dlm Z m Z d dl!m"Z" d dl#m$Z$ d dl%m&Z& d dl'm(Z( d d l)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6 d d
l7m8Z8 d dl9Z9d d l:m;Z;m<Z<m=Z=m>Z>m?Z?m@Z@mAZAmBZBmCZCmDZDmEZE d d lFmGZG d d
lHmIZImJZJ e+�r�d dlKmLZL daMe �NeO�ZPe jQdiZRdejS ejT ZUdZVdZWdd� ZXe$� dd� �ZYe$� dd� �ZZ�dde5e[e\f e[d�dd �Z]�dee5e[e\f e\d �d d �Z^e?�_d �e\e\d!�d"d#��Z`d$d%� ZaG d&d'� d'�ZbG d(d)� d)ec�ZdG d*d+� d+ec�Zed,d-� Zf�dfd.d/�Zg�dgd0d1�Zh�dhd2d3�Zi�did5d6�Zj�djd7d8�Zkdd9�e/d:�d;d<�Zld=d>� Zmd?d@� ZndAdB� Zo�dkdDdE�ZpdFdG� Zqe[erdH�dIdJ�Zse$� dKdL� �Zte$� dMdN� �Zue$� dOdP� �Zve$� dQdR� �Zwe$� dSdT� �Zxe$� dUdV� �Zy�dldXdY�Zz�dmdZd[�Z{�dnd\d]�Z|�dod^d_�Z}e$� d`da� �Z~dbdc� Ze$� ddde� �Z��dpdfdg�Z��dqdhdi�Z�djdk� Z��drdldm�Z��dse4e2 erdn�dodp�Z�ej dqdr� �Z�ej dsdt� �Z�dudv� Z�dwdx� Z��dtd{d|�Z��dud}d~�Z�derffdd��Z��dvd�d��Z�dd9�erd:�d�d��Z�dd9�erd:�d�d��Z��dwd�d��Z��dxd�d��Z�d�d�� Z�e d�g d���Z��dyd�d��Z��dzd�d��Z�e?�_d��e�d:�d�d���Z�d�d�� Z�d�d�� Z�d�d�� Z�d�d�� Z��d{d�d��Z��d|d�d��Z��d}d�d��Z��d~d�d��Z��dd�d��Z��d�d�d��Z�d�d�� Z�ddWd��e5e[e j�f e3e-e�gdf e�e\d��d�d��Z�ddWd��e5e[e j�f e3e-e�gdf e�e[d��d�d��Z�e$� d�d�� �Z�d�d�� Z�e�d:�d�d��Z��d�d�d��Z��d�d�d��Z��d�d�d��Z�e/e[e,f e3e[ e1e3e[ dd�dĄZ�e/e[e,f dŜd�dDŽZ��d�e[e[dȜd�dʄZ�d�d̄ Z�d�d΄ Z�d�dЄ Z��d�d�dӄZ�e?�_dԡerffd�dք�Z�d�d Z��d�d�dڄZ�ej d�d܄ �Z�d�dބ Z��d�d�d�Z�d�d� Z�d�d� Z��d�d�d�Z�d�d� Z�d�d� Zd�d� Z�e$� d�d� �Z�d�d� Z�d�d� Z��d�dWd�e�e�dd��d�d��Z�d�d�� Z�d�d�� Z�e[e�d��d�d��Z�e[e�d���d �d�Z�e[e[d���d�d�Z�e[e[d���d�d�Z�e[e1e[ �d��d�d�Z�e?�_�d ��d�dCdd�d �e�e��d ��d
�d��Zʐd�d� Zːd��d�d�Z̐d��d�d�Z͐d��d�d�Zΐd�d� Zϐd�d � ZАd �d � Zѐd �d � Z�e$� �d!�d"� �ZӐd#�d$� ZԐd�e[e[e/e[e[f �d&��d'�d(�ZՐd)�d*� Zd+�d,� Zאd-�d.� Z�ePdWf�d/�d0�Zِd1�d2� Zڐd3�d4� Zېd5�d6� Zܐd7�d8� Zݐd��d9�d:�Z�ePdWf�d;�d<�Z�e[e��d=��d>�d?�Z�d@�dA� Z�dde j�jQf�dB�dC�Z�d��dE�dF�Z�dG�dH� Z�d��dI�dJ�Z�dK�dL� Z�dM�dN� Z�dO�dP� Z�dQ�dR� Z�d��dS�dT�Z�d��dV�dW�Z�dX�dY� Z�dZ�d[� Z�d��d\�d]�Z�d^er�d_��d`�da�Z�e e0de,df d:��db�dc��Z�dS (� � N)� b64decode)�deque�
namedtuple)�contextmanager�suppress)�ENOENT)� lru_cache)�Path)�
ModuleType)
�IO�
TYPE_CHECKING�Any�Callable�Deque�Dict� Generator�List�Mapping�Optional�Sequence�Union�cast)�parse) �features�importer�mergers�net� performance�settings�subp�
temp_utils�
type_utils�
url_helper�version)�logexc)� CFG_BUILTIN�PER_ONCE)�Paths�_z_-.())�true�1ZonZyes)Zoff�0�noZfalsec C s" t ttt�� j�d�d d� ��S )N�.� )�tuple�map�int�os�uname�release�split� r6 r6 �2/usr/lib/python3.9/site-packages/cloudinit/util.py�kernel_version_ s r8 c C s t j ddgdd�} | j�� S )z�Return the sanitized string output by `dpkg --print-architecture`.
N.B. This function is wrapped in functools.lru_cache, so repeated calls
won't shell out every time.
Zdpkgz--print-architectureT��capture)r �stdout�strip)�outr6 r6 r7 �get_dpkg_architecturec s r> c
s� ddddd�} i � z~t j ddgdd �}|j�� D ],}|�d
�\}}}|| v r0|�� � | | < q0� fd d �| �� D �}t|�r�t�d
d� |�� W nH t j
y� } z.t�d|� t dd� | �� D ��� W Y d }~n
d }~0 0 � S )N�codename� description�idr4 )ZCodenameZ DescriptionzDistributor IDZRelease� lsb_releasez--allTr9 �:c s g | ]}|� vr|�qS r6 r6 )�.0�k��datar6 r7 �
<listcomp>~ � z lsb_release.<locals>.<listcomp>z.Missing fields in lsb_release --all output: %s�,z#Unable to get lsb_release --all: %sc s s | ]}|d fV qdS )Z UNAVAILABLENr6 �rD �vr6 r6 r7 � <genexpr>� rI z lsb_release.<locals>.<genexpr>) r r; �
splitlines� partitionr< �values�len�LOG�warning�join�ProcessExecutionError�dict)Zfmapr= �line�fnamer( �val�missing�errr6 rF r7 rB n s, �� ,rB �utf-8)�blob�returnc C s t | t�r| S | j|d�S �N)�encoding)�
isinstance�str�decode)r] r` r6 r6 r7 �
decode_binary� s rd )�textr^ c C s t | t�r| S | j|d�S r_ )ra �bytes�encode)re r` r6 r6 r7 � encode_text� s rh zBase64 decoding)rG r^ c C sF t | t�stdt| � ��zt| dd�W S tjy@ | Y S 0 dS )z�base64 decode data
If data is base64 encoded bytes, return b64decode(data).
If not, return data unmodified.
@param data: data as bytes. TypeError is raised if not bytes.
z data is '%s', expected bytesT)ZvalidateN)ra rf � TypeError�typer �binascii�ErrorrF r6 r6 r7 �maybe_b64decode� s
rm c C sP | j dd�}| �� dkrLt|t�rL| �� }|r<|jr<|j}nd}|�|d�S |S )NT)rc re r\ �surrogateescape)Z get_payloadZget_content_maintypera rf Z get_charsetZ input_codecrc )�partZ cte_payload�charsetr` r6 r6 r7 �fully_decoded_payload� s �
rq c @ s& e Zd Zd dd�Zdd� Zdd� ZdS )
� SeLinuxGuardFc C s> | zt �d�| _W n ty, d | _Y n0 || _|| _d S )N�selinux)r �
import_modulers � ImportError�path� recursive)�selfrv rw r6 r6 r7 �__init__� s zSeLinuxGuard.__init__c C s | j r| j �� rdS dS d S �NTF)rs �is_selinux_enabled)rx r6 r6 r7 � __enter__� s zSeLinuxGuard.__enter__c
C s� | j r| j �� sd S tj�| j�s&d S tj�| j�}z"t�|�}| j �||tj � W n t
yj Y d S 0 t � d|| j
� z| j j|| j
d� W n6 t
y� } z t �d|| j
|� W Y d }~n
d }~0 0 d S )Nz,Restoring selinux mode for %s (recursive=%s)�rw z,restorecon failed on %s,%s maybe badness? %s)rs r{ r2 rv �lexists�realpath�lstatZ matchpathcon�stat�ST_MODE�OSErrorrR �debugrw Z
restoreconrS )rx Z excp_typeZ
excp_valueZexcp_tracebackrv Zstats�er6 r6 r7 �__exit__� s0
��zSeLinuxGuard.__exit__N)F)�__name__�
__module__� __qualname__ry r| r� r6 r6 r6 r7 rr � s
rr c @ s e Zd ZdS )�MountFailedErrorN�r� r� r� r6 r6 r6 r7 r� � s r� c @ s e Zd ZdS )�DecompressionErrorNr� r6 r6 r6 r7 r� � s r� c O sv t �� }|dkr^z | |i |�� t �d� W qr tyZ ttdt�| �� t �d� Y qr0 nt�d|t�| �� d S )Nr z&Failed forking and calling callback %s� z(Forked child %s who will run callback %s) r2 �fork�_exit� Exceptionr$ rR r! �obj_namer� )Zchild_cb�args�kwargsZfidr6 r6 r7 �fork_cb� s" ��r� c C sB t | t�r| du S t}|r&t|�| }t| ��� �� |v r>dS dS rz )ra �bool� TRUE_STRINGS�listrb �lowerr< �rY �addonsZ check_setr6 r6 r7 �is_true s
r� c C sB t | t�r| du S t}|r&t|�| }t| ��� �� |v r>dS dS )NFT)ra r� �
FALSE_STRINGSr� rb r� r< r� r6 r6 r7 �is_false s
r� c C s | sdS t | t�r| S t| |�S )NF)ra r� r� )rY r� r6 r6 r7 �translate_bool s
r� � c s6 t �� � �stjtj �d�� �fdd�t| �D ��S )N� c s g | ]}� � ���qS r6 )�choice)rD Z_x��r� select_fromr6 r7 rH + rI z rand_str.<locals>.<listcomp>)�randomZ SystemRandom�string�
ascii_letters�digitsrT �range)�strlenr� r6 r� r7 �rand_str' s r� c C s* |sd}t dd�d | }|| vrq&q|S )Nr� � )r� r( )r� )Z
dictionaryZpostfixZnewkeyr6 r6 r7 �
rand_dict_key. s r� ��instance_data_file)r^ c
C s� ddl m}m}m}m} z t| �}W n ty: i Y S 0 |r�tj� |�r�z ||| |�}t
� d|| � W nx |y� } zt
� d| |� W Y d}~nPd}~0 |y� Y n8 |y� } z t
� d||t
|�� W Y d}~n
d}~0 0 t|i d�S )z>Read a yaml config with optional template, and convert to dictr )�JinjaLoadError�JinjaSyntaxParsingException�
NotJinjaError� render_jinja_payload_from_filez?Applied instance data in '%s' to configuration loaded from '%s'z4Failed to render templated yaml config file '%s'. %sNz:Could not apply Jinja template '%s' to '%s'. Exception: %s��default)Z!cloudinit.handlers.jinja_templater� r� r� r� �load_text_file�FileNotFoundErrorr2 rv �existsrR r� rS �repr� load_yaml)rX r� r� r� r� r� Z config_filer� r6 r6 r7 � read_conf8 sB
��� �r� c G s t t| � �S �N)�sorted�
uniq_merge)�listsr6 r6 r7 �uniq_merge_sortedl s r� c G sF g }| D ]4}t |t�r2|�� �d�}dd� |D �}|�|� qt|�S )NrJ c S s g | ] }|r|�qS r6 r6 )rD �ar6 r6 r7 rH ~ rI z uniq_merge.<locals>.<listcomp>)ra rb r< r5 �extend� uniq_list)r� Z
combined_list�a_listr6 r6 r7 r� x s
r� c C s` t �� D ]\}}| �||�} qg }| D ]}|tvr&|�|� q&|D ]}| �|d�} qB| �� } | S )Nr� )�FN_REPLACEMENTS�items�replace�
FN_ALLOWED�appendr< )�fnrE rL Zremovalsr6 r6 r7 �clean_filename� s r� Tc
C s z�t �t| ����}t�d dd|��d}|rRt|�� �W d � W d � W S |�� W d � W d � W S W d � n1 s�0 Y W d � n1 s�0 Y W nH t� y� } z.|r�| W Y d }~S tt |��|�W Y d }~n
d }~0 0 d S )N�rbr� )
�io�BytesIOrh �gzipZGzipFilerd �readr� r� rb )rG �quietrc �bufZghr� r6 r6 r7 � decomp_gzip� s �*fr� c C s~ | sdS | � dd�}|d �� }t|�dkr:|d �� }nd }|rV|dksV|�� dkrZd }|rr|dksr|�� dkrvd }||fS )N)NNrC r� r r. z-1Znone)r5 r< rQ r� )Zug_pairZ ug_parted�u�gr6 r6 r7 �extract_usergroup� s r� )�root_dirr^ c C sh t � }t�tj�| d��D ]H}tj�|�s,qtj�|�dd� }|�� }|r|�d�dkr|||< q|S )Nz*.pyr ���r- ���) rV �globr2 rv rT �isfile�basenamer< �find)r� �entriesrX �modnamer6 r6 r7 �get_modules_from_dir� s
r� c C s dt �� v S )a deprecated: prefer Distro object's `is_linux` property
Multiple sources of truth is bad, and already know whether we are
working with Linux from the Distro class. Using Distro offers greater code
reusablity, cleaner code, and easier maintenance.
ZLinux��platform�systemr6 r6 r6 r7 �is_Linux� s r� c C s$ dt �� v rdS t �� dkr dS dS )NZBSDTZ DragonFlyFr� r6 r6 r6 r7 �is_BSD� s
r� c C s t � d dkS )N�variant�freebsd�� system_infor6 r6 r6 r7 �
is_FreeBSD� s r� c C s t � d dkS )Nr� � dragonflyr� r6 r6 r6 r7 �is_DragonFlyBSD� s r� c C s t � d dkS )Nr� �netbsdr� r6 r6 r6 r7 � is_NetBSD� s r� c C s t � d dkS )Nr� �openbsdr� r6 r6 r6 r7 �
is_OpenBSD� s r� Fc C s || vr |S t | | �S r� )r� ��yobj�keyr� r6 r6 r7 �get_cfg_option_bool� s r� c C s* || vr |S | | }t |t�s&t|�}|S r� )ra rb )r� r� r� rY r6 r6 r7 �get_cfg_option_str� s
r� c C s t t| ||d��S )Nr� )r1 r� r� r6 r6 r7 �get_cfg_option_int� s r� c C s� | sd} t j�| �si S t| �}d}d|v r0d}t�||�}|r�|�� }d|d v r`|d |d< |d �� �d�d |d< |d d kr�d
|d< |d |d |d d �S i S )
z�Return a dictionary of distro info fields from /etc/redhat-release.
Dict keys will align with /etc/os-release keys:
ID, VERSION_ID, VERSION_CODENAME
z/etc/redhat-releasezA(?P<name>.+) release (?P<version>[\d\.]+) \((?P<codename>[^)]+)\)Z Virtuozzoz)(?P<name>.+) release (?P<version>[\d\.]+)�namer? z linuxr zred hat enterprise�redhatr# )�ID�
VERSION_ID�VERSION_CODENAME) r2 rv r� r� �re�match� groupdictr� rO )Z release_fileZredhat_releaseZ redhat_regexr� �groupr6 r6 r7 �_parse_redhat_release� s, � �r c C s� d} d}d}i }d}t j�d�r,ttd��}|s:d}t� }|r�|�dd�} |�dd�}d| v sfd| v rpt�� }nl| d ks�| d
kr�|�d d�}nN| d kr�|s�|�d d�}n4|�d
d�}|s�t �
d|�dd��}|r�|� � d }| dkr�d} n�t � �rt�
� �� } t�� }n�d}zPz t�� }W n t�y. Y n0 W d }|D ]}|�r:d}�q:|�s�t�d� n,d }|D ]}|�rfd}�qf|�s�t�d� 0 |S | ||fS )Nr� F�/etc/os-releaseTr� r� �sles�suse�alpine�photonZ PRETTY_NAME� virtuozzor� z [^ ]+ \((?P<codename>[^)]+)\)ZVERSIONr? �rhelr� )r� r� r� r� zPUnable to determine distribution, template expansion may have unexpected results)r2 rv r� �load_shell_contentr� r �getr� �machiner� r� r� r� r� r� r4 �distr� rR rS ) Z distro_nameZdistro_versionZflavorZ
os_releaseZos_release_rhelr� r �found�entryr6 r6 r7 �get_linux_distro sn
�
���r c C sv | d � � }d}|dkrf| d d � � }|dv r6|}qr|dv rDd}qr|d krRd
}qr|d v r`d }qrd}n |d
v rr|}|S )Nr� �unknownZlinuxr r )Z almalinuxr ZaoscZarchZ
azurelinuxZcentosZ
cloudlinuxZdebianZ eurolinuxZfedoraZmarinerZ miraclelinuxZ openeulerZ opencloudosZ openmandrivar r Zrockyr Z tencentosr )�ubuntuZ linuxmintZmintr r� r )Zopensusez
opensuse-leapzopensuse-microoszopensuse-tumbleweedZsle_hpcz sle-micror r )Zwindows�darwinr� r� r� r� )r� )�infor� r� Z
linux_distr6 r6 r7 � _get_variant] s r c C s< t � � t �� t �� t �� tt �� �t� d�} t| �| d< | S )N)r� r� r4 Zpythonr3 r r� )r� r� r4 Zpython_versionr� r3 r r )r r6 r6 r7 r� � s
� r� c C sX || vr |S | | du r g S | | }t |t�r@dd� |D �}|S t |t�sRt|�}|gS )a�
Gets the C{key} config option from C{yobj} as a list of strings. If the
key is present as a single string it will be returned as a list with one
string arg.
@param yobj: The configuration object.
@param key: The configuration key to get.
@param default: The default to return if key is not found.
@return: The configuration option as a list of strings or default if key
is not found.
Nc S s g | ]}|�qS r6 r6 rK r6 r6 r7 rH � rI z'get_cfg_option_list.<locals>.<listcomp>)ra r� rb )r� r� r� rY Zcvalr6 r6 r7 �get_cfg_option_list� s
r c C s> t |t�r|�d�}| }|D ] }||vr0| S || }q |S )a� Return the value of the item at path C{keyp} in C{yobj}.
example:
get_cfg_by_path({'a': {'b': {'num': 4}}}, 'a/b/num') == 4
get_cfg_by_path({'a': {'b': {'num': 4}}}, 'c/d') == None
@param yobj: A dictionary.
@param keyp: A path inside yobj. it can be a '/' delimited string,
or an iterable.
@param default: The default to return if the path does not exist.
@return: The value of the item at keyp."
is not found.�/)ra rb r5 )r� Zkeypr� Zcur�tokr6 r6 r7 �get_cfg_by_path� s
r c C s t | |�\}}t||� ||fS r� )�get_output_cfg�redirect_output)�cfg�mode�outfmt�errfmtr6 r6 r7 � fixup_output� s
r c
C s� t tj�d��r t�d� d S |s(tj}|s2tj}dd� }| �r t�d|| � | � dd�\}}|dksn|d kr�d
}|dkr~d }t
||�}n:|d kr�t j |d
t j
|d�} ttt | j�}n td| ��|r�t�|�� |�� � || k�r t�d|| � t�|�� |�� � d S |�r�t�d||� |� dd�\}}|dk�sD|d k�rbd
}|dk�rVd }t
||�}n<|d k�r�t j |d
t j
|d�} ttt | j�}n td| ��|�r�t�|�� |�� � d S )NZ_CLOUD_INIT_SAVE_STDOUTz5Not redirecting output due to _CLOUD_INIT_SAVE_STDOUTc S s< t �d� zt�d�j} W n ty, Y n 0 t �| � dS )a� Reconfigure umask and group ID to create output files securely.
This is passed to subprocess.Popen as preexec_fn, so it is executed in
the context of the newly-created process. It:
* sets the umask of the process so created files aren't world-readable
* if an adm group exists in the system, sets that as the process' GID
(so that the created file(s) are owned by root:adm)
� ZadmN)r2 �umask�grp�getgrnam�gr_gid�KeyError�setgid)Zgroup_idr6 r6 r7 � set_subprocess_umask_and_gid� s
z5redirect_output.<locals>.set_subprocess_umask_and_gidzRedirecting %s to %s� r� �>�>>�ab�wb�|T)�shell�stdinZ
preexec_fnz"Invalid type for output format: %sz!Invalid type for error format: %s)r� r2 �environr rR r� �sysr; �stderrr5 �open�
subprocess�Popen�PIPEr r r
r. ri �dup2�fileno)
r r Zo_outZo_errr& r �argZowithZnew_fp�procr6 r6 r7 r � sd
�
� r )�sourcesr^ c C sR |rt t| ��} i }| D ]4}|rt�|�}|s6t�� }t�|�}|�||�}q|S )aF Merge multiple dicts according to the dict merger rules.
Dict merger rules can be found in cloud-init documentation. If no mergers
have been specified, entries will be recursively added, but no values
get replaced if they already exist. Functionally, this means that the
highest priority keys must be specified first.
Example:
a = {
"a": 1,
"b": 2,
"c": [1, 2, 3],
"d": {
"a": 1,
"b": 2,
},
}
b = {
"a": 10,
"c": [4],
"d": {
"a": 3,
"f": 10,
},
"e": 20,
}
mergemanydict([a, b]) results in:
{
'a': 1,
'b': 2,
'c': [1, 2, 3],
'd': {
'a': 1,
'b': 2,
'f': 10,
},
'e': 20,
}
)r� �reversedr Zdict_extract_mergersZdefault_mergersZ construct�merge)r: �reverseZ
merged_cfgr Zmergers_to_applyZmergerr6 r6 r7 �
mergemanydict? s *
r> c c s8 t �� }z t �| � | V W t �|� n t �|� 0 d S r� )r2 �getcwd�chdir)ZndirZcurrr6 r6 r7 r@ w s
r@ c c s0 t �| �}z|V W t �|� n t �|� 0 d S r� )r2 r )Zn_msk�oldr6 r6 r7 r � s
r c C s dj | |d|d�S )Nz{0:{fill}{align}{size}}�^)�fillZalign�size)�format)re rC Zmax_lenr6 r6 r7 �center� s �rF c C s t �d| � t�| � d S )NzRecursively deleting %s)rR r� �shutil�rmtree�rv r6 r6 r7 �del_dir� s rJ r� � c
C s� z<t |||d�\}}}}|| d< || d< || d< || d< W dS tjy~ } z(|jtjkrhW Y d}~dS � W Y d}~n
d}~0 0 dS ) z�
returns boolean indicating success or failure (presense of files)
if files are present, populates 'fill' dictionary with 'user-data' and
'meta-data' entries
)�base�ext�timeout� user-data� vendor-data� meta-data�network-configTNF)� read_seededr" �UrlError�codeZ NOT_FOUND) rC rL rM rN �md�ud�vd�networkr� r6 r6 r7 �read_optional_seed� s rZ c C s� i }| sddg}n(t j�| �d�d�t j�| �d�d�g}t|�}dd� |D �}d }|D ],}t j�t j�|d��rXt j�|d�} q�qXd }|D ],}t j�t j�|d��r�t j�|d�} q�q�|r�|r�||d < ||d
< n |r�||d < |S ) Nz/var/lib/cloud/data/sslz /var/lib/cloud/instance/data/sslrG Zsslc S s g | ]}|rt j�|�r|�qS r6 )r2 rv �isdir)rD �dr6 r6 r7 rH � rI z%fetch_ssl_details.<locals>.<listcomp>zcert.pemzkey.pem� cert_file�key_file)r2 rv rT Z
get_ipath_cur� get_cpathr� r� )�pathsZ ssl_detailsZssl_cert_pathsr] r\ r^ r6 r6 r7 �fetch_ssl_details� s4 ��
ra c
C s" |}t | �} z\t�dt| �|� t�| �}|d u rBt�d� |}n t||�sbtd|t� |�f ��|}W n� tj
tt f�y } z�d}d }t |d�r�t
|d�r�t
|d�}n t |d�r�t
|d�r�t
|d�}|r�|dj|jd |jd |d �7 }n|d
j|d �7 }t�|� W Y d }~n
d }~0 0 |S ) NzKAttempting to load yaml from string of length %s with allowed root types %sz-loaded blob returned None, returning default.z2Yaml load allows %s root types, but got %s insteadzFailed loading yaml blobZ context_markZ problem_markz5. Invalid format at line {line} column {col}: "{err}"r� )rW �colr[ z. {err})r[ )rd rR r� rQ �yamlZ safe_loadra ri r! r� Z YAMLError�
ValueError�hasattr�getattrrE rW �columnrS )r] r� �allowedZloadedZ convertedr� �msgZmarkr6 r6 r7 r� � sF �
��
�� r� �
c
C s� | � d�dkrP| �dd| �}| �dd| �}| �dd| �}| �dd| �}nbtjrz| d dkrzt�| �jd krz| d7 } d
| d|f }d
| d|f }d
| d|f }d
| d|f }d }ztj|||d �} W n2 tj y� }
zt
� d |
� W Y d }
~
n d }
~
0 0 | � � �rt
| j�}tj|||d �} d } | � � �r>t
| ji d
�} tj|||d �}
d }|
� � �rb|
j}d }ztj|||d �}W n4 tj �y� }
zt
� d|
� W Y d }
~
n&d }
~
0 0 |� � �r�|j}n
t
� d� | |||fS )Nz%sr rO rP rQ rR r� r r� z%s%s%s)rN �retriesz No network config provided: %sr� z!Error in vendor-data response: %sz Error in vendor-data response)r� r� r Z%NOCLOUD_SEED_URL_APPEND_FORWARD_SLASHr �urlparse�queryr" Zread_file_or_urlrT rR r� �okr� �contents)rL rM rN rk Zud_urlZvd_urlZmd_urlZ network_urlrY Z network_respr� Zmd_resprV Zud_resprW rX Zvd_respr6 r6 r7 rS � sX �
"
�
�
�
"
rS c s� t t�� �dd�}dd� |D �}� fdd�|D �}g }|D ]x}tj�� |�}z|�t||d�� W q: ty� t� d|� Y q: t
y� } zt� d||� W Y d }~q:d }~0 0 q:t |�S )
z Read configuration directory.T�r= c S s g | ]}|� d �r|�qS )z.cfg)�endswith�rD �fr6 r6 r7 rH . rI z read_conf_d.<locals>.<listcomp>c s& g | ] }t j�t j�� |��r|�qS r6 )r2 rv r� rT rr ��confdr6 r7 rH 1 rI r� �1REDACTED config part %s, insufficient permissions� Error accessing file %s: [%s]N) r� r2 �listdirrv rT r� r� �PermissionErrorrR rS r� r> )ru r� Zconfs�cfgsr� rv r� r6 rt r7 � read_conf_d( s( �� �&r{ c
C s t � }i }zt| |d�}W nN ty8 t�d| � Y n< tyh } zt�d| |� W Y d}~nd}~0 0 |�|� d}d|v r�|d }|r�t|t�s�t d| t
� |�f ��q�t|�� � }n t
j�| � d��r�| � d�}|r�t
j�|�r�t||d�}|�|� t|�S ) a Read yaml file along with optional ".d" directory, return merged config
Given a yaml file, load the file as a dictionary. Additionally, if there
exists a same-named directory with .d extension, read all files from
that directory in order and return the merged config. The template
file is optional and will be applied to any applicable jinja file
in the configs.
For example, this function can read both /etc/cloud/cloud.cfg and all
files in /etc/cloud/cloud.cfg.d and merge all configs into a single dict.
r� rv rw Nr� Zconf_dz8Config file %s contains 'conf_d' with non-string type %sz.d)r r� ry rR rS r� r� ra rb ri r! r� r< r2 rv r[ r{ �
appendleftr> )Zcfgfiler� rz r r� ru Z confd_cfgr6 r6 r7 �read_conf_with_confdH s8 �$
��
r} c C s t t| d��S )N��cmdline)r� �read_cc_from_cmdliner~ r6 r6 r7 �read_conf_from_cmdlinew s r� c C s� | d u rt � } d| � �} d}d}t|�}t|�}t| �}g }| �|�}|dkr�| �||| �}|dk rj|}|�t�| || |� �� ��dd�� | �||| �}qFd�|�S )Nr' z cc:Zend_ccr z\n�
) � get_cmdlinerQ r� r� r �unquote�lstripr� rT ) r Z tag_beginZtag_endZbegin_lZend_lZclen�tokensZbegin�endr6 r6 r7 r� | s*
��r� c C s2 | � d�}|dks"| |d dkr&| S | �dd�S )Nr� r r� �
z
)r� r� )ro �posr6 r6 r7 �dos2unix� s
r� �HostnameFqdnInfo)�hostname�fqdn�
is_defaultc C s� d}d| v r,| d }t | d|�d�d �}nld| v rf| d �d�dkrf| d }| d d|�d�� }n2|jd|d�j}d| v r�| d }n|j|d �\}}t|||�S )
a� Get hostname and fqdn from config if present and fallback to cloud.
@param cfg: Dictionary of merged user-data configuration (from init.cfg).
@param cloud: Cloud instance from init.cloudify().
@param metadata_only: Boolean, set True to only query cloud meta-data,
returning None if not present in meta-data.
@return: a namedtuple of
<hostname>, <fqdn>, <is_default> (str, str, bool).
Values can be none when
metadata_only is True and no cfg or metadata provides hostname info.
is_default is a bool and
it's true only if hostname is localhost and was
returned by util.get_hostname() as a default.
This is used to differentiate with a user-defined
localhost hostname.
Fr� r� r- r NT)r� �
metadata_only)r� )r� r5 r� � get_hostnamer� r� )r �cloudr� r� r� r� r6 r6 r7 �get_hostname_fqdn� s �
�
r� �
/etc/hostsc C s� d}ztt |��� D ]b}|�d�}|dkr4|d|� }|�� }|sBq|�� }t|�dk rXq| |dd� v r|d } qvqW n ty� Y n0 |S )a�
For each host a single line should be present with
the following information:
IP_address canonical_hostname [aliases...]
Fields of the entry are separated by any number of blanks and/or tab
characters. Text from a "#" character until the end of the line is a
comment, and is ignored. Host names may contain only alphanumeric
characters, minus signs ("-"), and periods ("."). They must begin with
an alphabetic character and end with an alphanumeric character.
Optional aliases provide for name changes, alternate spellings, shorter
hostnames, or generic hostnames (for example, localhost).
N�#r � r. r� )r� rN r� r< r5 rQ �IOError)r� �filenamer� rW Zhashpos�toksr6 r6 r7 �get_fqdn_from_hosts� s$
r� z
Resolving URLc
C sX t �| �}|j}tdu r�t� }d}i }|D ]�}z`t�|dddtjtj�}g ||< |D ]6\}} }
} } || � d| | d f � |�
| d � qTW q* tj tj fy� Y q*0 q*|a|r�t
�d|� zttt��4 t�|j�d��r�W d � W dS W d � n1 �s0 Y t�|d�}|d d d }
|
tvW S tj tj f�yR Y d S 0 dS )
a� determine if a url's network address is resolvable, return a boolean
This also attempts to be resilent against dns redirection.
Note, that normal nsswitch resolution is used here. So in order
to avoid any utilization of 'search' entries in /etc/resolv.conf
we have to append '.'.
The top level 'invalid' domain is invalid per RFC. And example.com
should also not exist. The '__cloud_init_expected_not_found__' entry will
be resolved inside the search list.
N)zdoes-not-exist.example.com.zexample.invalid.Z!__cloud_init_expected_not_found__r z%s: %sz detected dns redirection: %sz[]T� F)r rl r� �_DNS_REDIRECT_IP�set�socketZ getaddrinfoZ SOCK_STREAMZ AI_CANONNAMEr� �addZgaierror�errorrR r� r rd r Z
is_ip_address�netlocr< )�urlZ
parsed_urlr� ZbadipsZbadnamesZ
badresultsZiname�resultZ_famZ_stypeZ_protoZcnameZsockaddr�addrr6 r6 r7 �
is_resolvable s:
�
2
r� c C s t �� } | S r� )r� Z gethostname)r� r6 r6 r7 r� 4 s r� c C s, zt �| �d W S t jy& Y d S 0 d S �Nr )r� �
gethostbyaddrZherror)�ipr6 r6 r7 r� 9 s r� c C s t | �S )z5determine if this url is resolvable (existing or ip).)r� )r� r6 r6 r7 �is_resolvable_url@ s r� c C s\ | du r dS t �d| � | D ]:}z"t|�r@t �d|� |W S W q tyT Y q 0 q dS )zc
Search through a list of mirror urls for one that works
This needs to return quickly.
Nz%search for mirror in candidates: '%s'zfound working mirror: '%s')rR r� r� r� )Z
candidatesZcandr6 r6 r7 �search_for_mirrorE s r� �devicec C sv g }| s t � d�t � d� S | �d�rL| �d�}dd� d| d| fD �}n&| dkr`t � d�}n| d krrt � d�}|S )
Nz/dev/msdosfs/*z/dev/iso9660/*�LABEL=c S s g | ]}t j�|�r|�qS r6 �r2 rv r� )rD �pr6 r6 r7 rH ` s �z*find_devs_with_freebsd.<locals>.<listcomp>z
/dev/msdosfs/z
/dev/iso9660/� TYPE=vfat� TYPE=iso9660)r� �
startswithr� )�criteria�oformat�tag�no_cacherv �devlist�labelr6 r6 r7 �find_devs_with_freebsdX s
�
r� c C s� g }d }d }d}| r<| � d�r(| �d�}| � d�r<| �d�}tjg d�dgd�} | j�� D ]j}
|sf|r�tjd|
gddgd�\}} |r�d | |vr�qZ|d
kr�d |vr�qZ|d kr�d |v r�qZ|�d
|
� qZ|S )Nr� r� zTYPE=��sysctl�-nz hw.disknamesr �ZrcsZ mscdlabelr� z
label "%s"�iso9660zISO filesystem�vfat�/dev/)r� r� r r; r5 r� ) r� r� r� r� rv r� r� �_typeZ
mscdlabel_outr= �devr( r6 r6 r7 �find_devs_with_netbsdl s*
r� c C s� t j g d�dgd�}g }|j�� �d�D ]P}|�d�s8q(|dkrBq(|�|d d� d � |�d �s(|�|d d� d
� q(d d � |D �S )
Nr� r r� rJ rC zfd0:r� r� �cd�ic S s g | ] }d | �qS �r� r6 �rD r� r6 r6 r7 rH � rI z*find_devs_with_openbsd.<locals>.<listcomp>)r r; �rstripr5 rq r� r� )r� r� r� r� rv r= r� r
r6 r6 r7 �find_devs_with_openbsd� s
r� c C s~ t j g d�dgd�}dd� t|j�� dd�D �}| dkrHd d� |D �}n(| d
v r`d d� |D �}n| rpt�d | � d
d� |D �S )N)r� r� z
kern.disksr r� c S s$ g | ] }|� d �s|� d�s|�qS )rV Zvn�r� r� r6 r6 r7 rH � s �z/find_devs_with_dragonflybsd.<locals>.<listcomp>Trp r� c S s g | ]}|� d �r|�qS �)r� Zacdr� r� r6 r6 r7 rH � rI )zLABEL=CONFIG-2r� c S s g | ]}|� d �s|�qS r� r� r� r6 r6 r7 rH � rI zUnexpected criteria: %sc S s g | ] }d | �qS r� r6 r� r6 r6 r7 rH � rI )r r� r; r5 rR r� )r� r� r� r� rv r= r� r6 r6 r7 �find_devs_with_dragonflybsd� s � r� c
C sH t � rt| ||||�S t� r,t| ||||�S t� rBt| ||||�S t� rXt| ||||�S dg}g }| rt|�d| � |r�|�d| � |r�|� ddg� |r�|�d| � |r�|�|� || }zt
j
|ddgd �\}} W n< t
j �y }
z |
j t
k�r d
}n� W Y d }
~
n
d }
~
0 0 g } |�� D ] } | �� } | �r$| �| � �q$| S ) z�
find devices matching given criteria (via blkid)
criteria can be *one* of:
TYPE=<filesystem>
LABEL=<label>
UUID=<uuid>
�blkidz-t%sz-s%s�-c� /dev/nullz-o%sr r. r� r� N)r� r� r� r� r� r� r� r� r� r� r rU �errnor rN r< )
r� r� r� r� rv Z
blk_id_cmd�options�cmdr= �_errr� r� rW r6 r6 r7 �find_devs_with� sF
�
r� c C s� | du rg } nt | �} g d�}|r0|�ddg� |�| � tj|ddd�}i }|j�� D ],}|�d�\}}}t|�||< ||| d < qX|S )
z�Get all device tags details from blkid.
@param devs: Optional list of device paths you wish to query.
@param disable_cache: Bool, set True to start with clean cache.
@return: Dict of key value pairs of info for the device.
N)r� �-oZfullr� r� Tr� )r: rc rC ZDEVNAME)r� r� r r; rN rO r ) ZdevsZ
disable_cacher� r= �retrW r� r( rG r6 r6 r7 r� � s
r� c C s( g }| D ]}||v rqq|� |� q|S r� )r� )Zin_listZout_listr� r6 r6 r7 r� s r� ��read_cbr� )rX r� r� r^ c
C s� t jddd�}t�d| |� |�� t�� �n}z<t| d�� }t|||d� W d � n1 s\0 Y W n ty� |s|� Y n0 |� � }W d � n1 s�0 Y W d � n1 s�0 Y t�dt
|�| |j � |S )Nr� �skip)Zlog_modezReading from %s (quiet=%s)r� )�chunk_cbzReading %s bytes from %s%s) r ZTimedrR r� r� r� r2 � pipe_in_outr� �getvaluerQ �output)rX r� r� ZtimerZofhZifhro r6 r6 r7 �load_binary_file s
0 Dr� c C s t t| ||d��S )Nr� )rd r� )rX r� r� r6 r6 r7 r� ( s r� c
C s� t � r^z td�} | �dd�d d� }W q� tyZ } z t�d|� d}W Y d }~q�d }~0 0 n(ztd��� }W n ty� d}Y n0 |S )Nz/proc/1/cmdline� r' r� z"failed reading /proc/1/cmdline: %sr� z
/proc/cmdline)� is_containerr� r� r� rR rS r< )ro r r� r6 r6 r7 � _get_cmdline1 s
r� c C s dt jv rt jd S t� S )NZDEBUG_PROC_CMDLINE)r2 r/ r� r6 r6 r6 r7 r� D s
r� c C s8 d} zt | ��� }|dkW S ttfy2 Y dS 0 d S )Nz /proc/sys/crypto/fips_enabledr* F)r� r< r� r� )Z fips_procro r6 r6 r7 � fips_enabledK s
r� � c C sN d}| � |�}t|�dkr qBq|�|� |t|�7 }|r||� q|�� |S r� )r� rQ �write�flush)Zin_fhZout_fhZ
chunk_sizer� Z bytes_pipedrG r6 r6 r7 r� W s
r� c C s6 |dv r|dv rd S t �d| ||� t�| ||� d S )N)Nr� z%Changing the ownership of %s to %s:%s)rR r� r2 �chown)rX �uid�gidr6 r6 r7 � chownbyidf s r� c
C sp d}d}z$|rt �|�j}|r*t�|�j}W n2 ty^ } ztd| �|�W Y d }~n
d }~0 0 t| ||� d S )Nr� zUnknown user or group: %s) �pwd�getpwnam�pw_uidr! r" r# r$ r� r� )rX �userr� r� r� r� r6 r6 r7 � chownbynamen s $r� )r r r^ c C sh ddg}| rd| vr|S | d }||v r2|| }nd|vr>|S |d }t |t�rX||g}t |t�r�t|�dkrz|d |d< t|�dkr�|d |d< t |t�r�d|v r�|d |d< d|v r�|d |d< |d dkr�|d |d< g d�}t|�D ]v\}}|s�q�|�� }d } |D ]8}
|�|
��r
d
|
|t|
�d� �� f }d } �qD�q
| �sZd
d |�� f }|||< q�|S )
a� Get the output configuration for a given mode.
The output config is a dictionary that specifies how to deal with stdout
and stderr for the cloud-init modules. It is a (frustratingly) flexible
format that can take multiple forms such as:
output: { all: "| tee -a /var/log/cloud-init-output.log" }
or
output:
init:
output: "> /var/log/cloud-init.out"
error: "> /var/log/cloud-init.err"
config: "tee -a /var/log/cloud-config.log"
final:
- ">> /var/log/cloud-final.out"
- "/var/log/cloud-final.err"
Mode can be one of the configuration stages. If you pass a
non-existent mode, it will assume the "all" mode configuration if
defined.
Stderr can be specified as &1 to indicate that it should
be the same as stdout.
If a file is specified with no redirection, it will default to
appending to the file.
If not overridden, output is provided in
'/etc/cloud/config/cloud.cfg.d/05_logging.cfg' and defaults to:
{"all": "| tee -a /var/log/cloud-init-output.log"}
:param cfg: The base configuration that may or may not contain the
'output' configuration dictionary
:param mode: The mode to get the output configuration for.
:return: A list of two strings (or Nones), the first for stdout for the
specified mode and the second for stderr.
Nr� �allr r� r� z&1)r) r( r, Fz%s %sTr) ) ra rb r� rQ rV � enumerater� r� r< ) r r r� ZoutcfgZmodecfgZswlistr� r� rY r �sr6 r6 r7 r { sL '
r )r c
C s� g }g }| rt | t�s|S | �d�}|r2|�|� t| d�D ]h}|sFq<t�d|�}|sXq<|�d�}|�� }t |�dkr�|�|� q<ddg|dd� kr<|�|d � q<|D ]8}t
�
|� d ��D ]"} t j �
| �r�| |kr�|�| � q�q�tt|| ��S )
a= Return a list of log file paths from the configuration dictionary.
Obtains the paths from the 'def_log_file' and 'output' configuration
defined in the base configuration.
If not provided in base configuration, 'def_log_file' is specified in
'cloudinit/settings.py' and defaults to:
/var/log/cloud-init.log
If not overridden, output is provided in
'/etc/cloud/config/cloud.cfg.d/05_logging.cfg' and defaults to:
{"all": "| tee -a /var/log/cloud-init-output.log"}
@param cfg: The cloud-init merged configuration dictionary.
Z def_log_fileNz (?P<type>\||>+)\s*(?P<target>.*)�targetr� �teez-ar. �*)ra rV r r� r r� r� r� r5 rQ r� r2 rv r� r� r� )
r ZlogsZ rotated_logsZ default_log�fmtr� r� �partsZlogfileZrotated_logfiler6 r6 r7 �get_config_logfiles� s0
r� )�routiner^ c C s< t �|�}|�t| �� |�� }|d ur4|d|� S |S d S r� )�hashlib�new�updaterh Z hexdigest)r] r� �mlenZhasherZdigestr6 r6 r7 � hash_blob s
r� c C s. zt �| �rW dS W n ty( Y dS 0 d S rz )r� r� r$ �r� r6 r6 r7 �is_user s
r� c C s. zt �| �rW dS W n ty( Y dS 0 d S rz )r! r" r$ r� r6 r6 r7 �is_group s
r� c C s t �d| |� t�| |� d S )NzRenaming %s to %s)rR r� r2 �rename��src�destr6 r6 r7 r� & s r� �� c C s | D ]}t ||� qd S r� )�
ensure_dir)�dirlistr r\ r6 r6 r7 � ensure_dirs, s r z Loading jsonc C sH t �t| ��}t|t|��sDd�dd� |D ��}td|t|�f ��|S )N�, c S s g | ] }t |��qS r6 )rb )rD �tr6 r6 r7 rH 5 rI z load_json.<locals>.<listcomp>z((%s) root types expected, got %s instead)�json�loadsrd ra r/ rT ri rj )re Z
root_typesZdecodedZexpected_typesr6 r6 r7 � load_json1 s
��r c C s< t j�| �}|t j�|�kr | S t j�|�r0| S t|�S dS )z�Get the last directory in a path that does not exist.
Example: when path=/usr/a/b and /usr/a does not exis but /usr does,
return /usr/a
N)r2 rv �dirnamer[ �get_non_exist_parent_dir)rv �p_pathr6 r6 r7 r
= s r
c
C s� t j�| �s�t| �}t j�| �}d}|dkr0d}t||d�� t �| � W d � n1 s\0 Y t| |� |sx|r�t|||� t | �
|�d dd � �}t |�}|j D ]} |� | �}t|||� q�n
t| |� d S )NTr Fr} r� )
r2 rv r[ r
r rr �makedirs�chmodr� r r5 r� �joinpath)
rv r r� r� Znon_existed_parent_dirZdir_nameZselinux_recursiveZsub_relative_dirZsub_pathro r6 r6 r7 r N s" (
r c c s> z | V W | r:d| g}t � |� n| r8d| g}t � |� 0 d S )N�umount)r )r Z
umount_cmdr6 r6 r7 � unmounteri s �r c C s i } z�t j�d�r$td��� }d}nt�d�}|j�� }d}t�d�}|D ]�}|dkr~|� � }t
|�dkrlqJ|\}}} }
} } nL|� |�}
|
d u sJt
|
� � �dk r�qJ|
�
d�}|
�
d�}|
�
d �} |
�
d�}
|�d
d �}| ||
d �| |< qJt�d
| |� W n" ttf�y ttd� Y n0 | S )Nz /proc/mountsr9 �mountz*^(/dev/[\S]+) on (/.*) \((.+), .+, (.+)\)$� r� r� r. r� z\040r' )�fstype�
mountpoint�optszFetched %s mounts from %sz Failed fetching mount points)r2 rv r� r� rN r r; r� �compiler5 rQ �search�groupsr� r� rR r� r� r� r$ )�mountedZ
mount_locs�methodr= ZmountreZmpline�wordsr� Zmpr r Z_freqZ_passno�mr6 r6 r7 �mountss s>
� r c
C sJ t |t�r|g}n:t |ttf�r*t|�}n"|du r8d}ntdjt|�d���t� rb|du r�dg}nPt� r�|du rxg d�}t |�D ](\}}|dkr�d||< |dv r�d ||< q�nd
g}t
� }t � � ��r} d }
t
j�| �|v r�|t
j�| � d } n�d} |D ]�}d} zRg d
�}
|�r |
�d|g� |
�| � |
�| � tj|
|d� | }
| } W �q�W q� ttf�y� } z.|�r�t�d| |d�|
�|� |} W Y d}~q�d}~0 0 q�| �s�td| | | f ��| �d��s�| d7 } t|
��@ |du �r�|| �}n
|| |�}|W d � W d � S 1 �s 0 Y W d � n1 �s<0 Y dS )a2
Mount the device, call method 'callback' passing the directory
in which it was mounted, then unmount. Return whatever 'callback'
returned. If data != None, also pass data to callback.
mtype is a filesystem type. it may be a list, string (a single fsname)
or a list of fsnames.
Nz6Unsupported type provided for mtype parameter: {_type})r� �auto)�ufs�cd9660�msdosr� r# )r� Zmsdosfsr$ r� Fr )r r� �roz-t)Z
update_envzbFailed to mount device: '%s' with type: '%s' using mount command: '%s', which caused exception: %sr' z#Failed mounting %s to %s due to: %sr )ra rb r� r/ ri rE rj r� r� r� r r Ztempdirr2 rv r r� r� r r� r� rR r� rT r� rq r )r� �callbackrG ZmtypeZupdate_env_for_mountZ log_errorZmtypes�indexr Ztmpdr r Zfailure_reasonZmountcmd�excr� r6 r6 r7 �mount_cb� s�
��
� ��
r) c C s
t �t�S r� )�obj_copy�deepcopyr% r6 r6 r6 r7 �get_builtin_cfg� s r, c C s t �d| � tj�| �S )Nz Testing if a link exists for %s)rR r� r2 rv �islinkrI r6 r6 r7 �is_link s r. c C sh t �d|| � |rXtj�|�rXtj�tj�|�dtd� �}t�| |� t� ||� d S t�| |� d S )Nz$Creating symbolic link from %r => %r�tmpr� )
rR r� r2 rv r~ rT r r� �symlinkr� )�source�link�forceZtmp_linkr6 r6 r7 �sym_link s r4 c C s2 t �d| � zt�| � W n ty, Y n0 d S )NzAttempting to remove %s)rR r� r2 �unlinkr� rI r6 r6 r7 �del_file s
r6 c C s t �d| |� t�| |� d S )NzCopying %s to %s)rR r� rG �copyr r6 r6 r7 r7 s r7 c C s0 zt �dt �� �} W n ty* d} Y n0 | S )Nz%a, %d %b %Y %H:%M:%S %z�??)�time�strftime�gmtimer� )�tsr6 r6 r7 � time_rfc2822 s
r= c s� ddl � ddl� G � fdd�d� j�} � �� j�d��}� �� }� �| �|_dt � sXdndg}� j
d |� }| � }|� |� �
t |��� �
|�� �
|�dd�d
kr�|j|jd S td ��dS )
a� Use sysctl(3) via ctypes to find kern.boottime
kern.boottime is of type struct timeval. Here we create a
private class to easier unpack it.
Use sysctl(3) (or sysctl(2) on OpenBSD) because sysctlbyname(3) does not
exist on OpenBSD. That complicates retrieval on NetBSD, which #defines
KERN_BOOTTIME as 83 instead of 21.
21 on NetBSD is KERN_OBOOTTIME, the kern.boottime up until NetBSD 5.0
@return boottime: float to be compatible with linux
r Nc s e Zd Zd� jfd� jfgZdS )zboottime.<locals>.timeval�tv_sec�tv_usecN)r� r� r� Zc_int64Z_fields_r6 ��ctypesr6 r7 �timeval6 s rB �cr� � �S r. r� g ��.Az/Unable to retrieve kern.boottime on this system)rA Z ctypes.utilZ StructureZCDLL�utilZ find_libraryZc_size_tZsizeof�valuer� Zc_intr� rQ Zbyrefr> r? � RuntimeError)rB ZlibcrD Z
mib_valuesZmibr� r6 r@ r7 �boottime& s0
� ��� rI c C sr d} d}zDt j�d�r4d}td�}|rJ|�� d } nd}tt�� t� �} W n tyl t t
d| � Y n0 | S )Nr8 r z /proc/uptimer rA z&Unable to read uptime using method: %s) r2 rv r� r� r5 rb r9 rI r� r$ rR )Z
uptime_strr ro r6 r6 r7 �uptimeQ s rJ c C s t | |dd d� d S )Nr* )�omoder ��
write_file)rv �contentr6 r6 r7 � append_filed s rO � )�
preserve_mode)r rQ r^ c C s t | dd||d� d S )Nr� r* )rN rK r rQ rL )rv r rQ r6 r6 r7 � ensure_fileh s
�rR c C s( z
t | �W S ttfy" Y d S 0 d S r� )r1 rd ri )Z possible_intr6 r6 r7 �safe_intp s
rS c C sH t |�}| rD|rDt| �� t�| |� W d � n1 s:0 Y d S r� )rS rr r2 r )rv r Z real_moder6 r6 r7 r w s
r )�grp_namer^ c C s8 d}zt �| �j}W n ty2 t�d| � Y n0 |S )zt
Returns the group id of a group name, or -1 if no group exists
@param grp_name: the name of the group
r� z"Group %s is not a valid group name)r! r" r# r$ rR r� )rT r� r6 r6 r7 � get_group_id~ s rU )rv r^ c C s t �t� | �j�S )z�
Returns the octal permissions of the file/folder pointed by the path,
encoded as an int.
@param path: The full path of the file/folder.
)r� �S_IMODEr2 �st_moderI r6 r6 r7 �get_permissions� s rX c C s t �| �}t�|j�jS )zw
Returns the owner of the file/folder pointed by the path.
@param path: The full path of the file/folder.
)r2 r� r� �getpwuid�st_uid�pw_name�rv �str6 r6 r7 � get_owner� s
r^ c C s t �| �}t�|j�jS )zw
Returns the group of the file/folder pointed by the path.
@param path: The full path of the file/folder.
)r2 r� r! �getgrgid�st_gid�gr_namer\ r6 r6 r7 � get_group� s
rb )�usernamer^ c C sJ g }t �� D ]}| |jv r |�|j� q t�| �j}|�t �|�j� |S )zp
Returns a list of all groups to which the user belongs
@param username: the user we want to check
) r! �getgrall�gr_memr� ra r� r� �pw_gidr_ )rc r r� r� r6 r6 r7 �get_user_groups� s
rg z Writing filer+ )�ensure_dir_existsr� r� )rQ rh c C s |r$z t | �}W n ty" Y n0 |r>ttj�| �||d� d|�� v rXt|�}d}n t|�}d}z d| } W n t y� d| } Y n0 t
� d| || t |�|� t
| d��L t| |��"}
|
�|� |
�� W d � n1 s�0 Y W d � n1 �s0 Y t| |� d S )
a)
Writes a file with the given content and sets the file mode as specified.
Restores the SELinux context if possible.
@param filename: The full path of the file to write.
@param content: The content to write to the file.
@param mode: The filesystem mode to set on the file.
@param omode: The open mode used when opening the file (w, wb, a, etc.)
@param preserve_mode: If True and `filename` exists, preserve `filename`s
current mode instead of applying `mode`.
@param ensure_dir_exists: If True (the default), ensure that the directory
containing `filename` exists before writing to
the file.
@param user: The user to set on the file.
@param group: The group to set on the file.
)r� r� �brf Z
charactersz%oz%rz Writing to %s - %s: [%s] %s %srI N)rX r� r r2 rv r r� rh rd ri rR r� rQ rr r2 r� r� r ) r� rN r rK rQ rh r� r� Z
write_typeZmode_rZfhr6 r6 r7 rM � s: �
FrM c C s@ t �| �D ]0}t j�| |�}t j�|�r2t|� q
t|� q
dS )z�
Deletes all contents of a directory without deleting the directory itself.
@param dirname: The directory whose contents should be deleted.
N)r2 rx rv rT r[ rJ r6 )r ZnodeZ
node_fullpathr6 r6 r7 �delete_dir_contents� s
rj r� �createdc C s6 t �� }t| �}|d|�� |f 7 }|dt� 7 }|S )Nz %s by cloud-init v. %sz on %s)r# Zversion_stringrb �titler= )Z comment_charrL Zci_ver�headerr6 r6 r7 � make_header s
rn c C s� t | ttf�s tdt�| � ��d}|r0|d7 }dd }d}| D ]�}t |ttf�r�g }|D ] }|�dt|��d|� � qZd |d
� |�f }|d 7 }q@t |t�r�d ||f }|d 7 }q@|d u r�q@td t�|�|f ��q@t
� d
|� |S )Nz8Input to shellify was type '%s'. Expected list or tuple.r� z
#!/bin/sh
z%s%s%s%s)�'�\ro ro r z'%s'ro z%s%s
r' r� zCUnable to shellify type '%s'. Expected list, string, tuple. Got: %szShellified %s commands.) ra r/ r� ri r! r� r� rb r� rT rR r� )ZcmdlistZ
add_headerrN ZescapedZ cmds_mader� �fixedrs r6 r6 r7 �shellify s< ��
�� rr c C sB |r | � |�r | t|�d � } |r>| �|�r>| d t|� � } | S r� )r� rQ rq )rW �prefix�suffixr6 r6 r7 �strip_prefix_suffix8 s
ru c C s@ t �| d �d u rdS zt � | � W n t jy: Y dS 0 dS )Nr FT)r �whichrU )r� r6 r6 r7 �_cmd_exits_zero@ s rw c C s t g d��S )N)zsystemd-detect-virtz--quietz --container�rw r6 r6 r6 r7 �_is_container_systemdJ s ry c C s
t dg�S )Nzlxc-is-containerrx r6 r6 r6 r7 �_is_container_old_lxcN s rz c C sB t � s
dS g d�} t�| d �d u r(dS t�| �\}}|�� dkS )NF)r� z-qnzsecurity.jail.jailedr r* )r� r rv r< )r� r= r( r6 r6 r7 �_is_container_freebsdR s r{ c C s� t ttf} | D ]}|� r dS qz(td�}d|v r8W dS d|v rFW dS W n ttfy^ Y n0 tj�d�r|tj�d�s|dS zHt d��
� }|D ]2}|� d�r�|� � �
d d�\}}|d
kr� W dS q�W n ttfy� Y n0 d S ) zH
Checks to see if this code running in a container of some sort
Tr� � containerZLIBVIRT_LXC_UUIDz/proc/vzz/proc/bcz/proc/self/statuszVxID:rC r+ F)ry r{ rz � get_proc_envr� r� r2 rv r[ r� rN r� r< r5 )Zchecks�helperZpid1env�linesrW Z_keyrY r6 r6 r7 r� \ s6 �
r� c C s t j�d�S )z2Check to see if we are running in a lxd container.z
/dev/lxd/sockr� r6 r6 r6 r7 �is_lxd� s r� r� )r` �errorsr^ c C s� t j�dt| �d�}z t|�}W n ttfy: i Y S 0 i }d\}}|�||�}|�|�D ]&}|shq^|�|d�\} }
| r^|
|| < q^|S )z�
Return the environment in a dict that a given process id was started with.
@param encoding: decoding will be done with .decode(encoding, errors) and
text will be returned.
@param errors: passed through .decode(encoding, errors).
z/procr/ )r� �=r� ) r2 rv rT rb r� r� r� rc r5 ) �pidr` r� r� ro �envZnullZequalr r� rY r6 r6 r7 r} � s
r} c C sN i }| � � D ]<}z|� dd�\}}W n ty> |}d}Y n0 |||< q |S )Nr� r� T)r5 rd )Zkvstringr� r r� rY r6 r6 r7 �keyval_str_to_dict� s
r� c C s& | � d�r| dd � } tj�d| �S )Nr� rK z /sys/class/block/%s/partition)r� r2 rv r� )r� r6 r6 r7 � is_partition� s
r� c C s� t |t�s|g}g }|D ]�}t |t�r2|�|� qt |ttf�r�t|�dk sXt|�dkr`td��t|�dkr�|d r�|�| t|� � q|�|d � qtd��q|S )Nr� r. z Invalid package & version tuple.r zInvalid package type.)ra r� rb r� r/ rQ rH )Z version_fmtZpkgsZpkglist�pkgr6 r6 r7 �expand_package_list� s
r� c C s� dd� | � d�D �}d}d}d}d}t|�D �]@\} }
|
� � } t| �dk rh|�d| d t| �|
� dS | d } d d� | � d�D �}
t|
�t|�kr�q,tt|
�t|��}|
d
|� |d
|� kr�q,|dur�t|�t|
�kr�q,z| �d �} W n* t�y |�d | d |
� Y dS 0 z | | d }| | d
}W n* t�y\ |�d| d |
� Y dS 0 | }|
}| d }q,|�r�|�r�|�r�|�r�|�r�||||fS n |�r�|�r�|�r�|||fS dS )zRReturn the mount information for PATH given the lines from
/proc/$$/mountinfo.c S s g | ] }|r|�qS r6 r6 �rD r� r6 r6 r7 rH � rI z$parse_mount_info.<locals>.<listcomp>r Nrj z$Line %d has two few columns (%d): %sr� r� c S s g | ] }|r|�qS r6 r6 r� r6 r6 r7 rH � rI r �-z,Did not find column named '-' in line %d: %sr. z/Too few columns after '-' column in line %d: %srK )r5 r� rQ r� �minr' rd �
IndexError)rv Zmountinfo_lines�log� get_mnt_opts�
path_elements�devpth�fs_type�match_mount_point�match_mount_point_elementsr� rW r� � mount_point�mount_point_elements�x�
mount_optionsr6 r6 r7 �parse_mount_info� sb �
��
�
�
r� c C sB t d��� D ]0}|�� dd� \}}}|| kr |||f S q dS )z<On older kernels there's no /proc/$$/mountinfo, so use mtab.� /etc/mtabNr� )r� rN r5 )rv rW r� r� r� r6 r6 r7 �
parse_mtab*
s
r� c C s� | � d�}t|�dkr |d S t|�dkr2|d S |d dv r�| dd � }t�g d��\}}|� d �D ]0}|� � }t|�dkrf|d |krf|d } q�qft|�S t�d
| � d S ) Nr r� r r� r. )r� ZgptZgptidr" ZufsidrK )Zglabel�statusz-sr� z)Unexpected input in find_freebsd_part: %s)r5 rQ r rb rR rS )ZfsZsplittedZ target_labelro r� �labelsr� r6 r6 r7 �find_freebsd_part3
s
r� c C sH d }|� d�D ]4}|� � }t|�dkrtj�|d | �r|} qDq|S )Nr� r. r� )r5 rQ r2 rv r� )rv �mnt_list�
path_foundrW r� r6 r6 r7 �get_path_dev_freebsdF
s r� c C sl t j dd| gddgd�\}}t|�rPt � ddg�\}}t| |�}|d u rLd S |}|�� }t|d �}d| S )Nr z-pr r� r� r� )r rQ r� r5 r� )rv r� r[ r� r� r� Z
label_partr6 r6 r7 �get_freebsd_devpthP
s
r� c C s� t � dg�\}}d}dd� | �d�D �}d}d}d}d} |�� D �],}
t�||
�} | sXq@| �d�}| �d�}d d� |�d�D �} t| �t|�kr�q@tt| �t|��}
| d
|
� |d
|
� kr�q@| dur�t| �t| �kr�q@|}| } | �d �}| �d �}|du�r d
�| �d �� d
�� � �d��}t
� d||||� t�d|�}|�s\t � �r\|dv�r\t
|�}|}|| kr@ �qpq@|�r�|| v�r�dS |�r�|�r�|�r�|�r�|�r�||||fS n |�r�|�r�|�r�|||fS dS )z�Return the mount information for PATH given the lines ``mount(1)``
This function is compatible with ``util.parse_mount_info()``r ze^(?P<devpth>[\S]+?) on (?P<mountpoint>[\S]+?) (\(|type )(?P<type>[^,\(\) ]+)( \()?(?P<options>.*?)\)$c S s g | ] }|r|�qS r6 r6 r� r6 r6 r7 rH k
rI z parse_mount.<locals>.<listcomp>r Nr� r c S s g | ] }|r|�qS r6 r6 r� r6 r6 r7 rH v
rI r rj r� rJ r zNfound line in mount -> devpth: %s, mount_point: %s, fs_type: %s, options: '%s'z^(/dev/.+)[sp]([0-9])$)Zzfs�nfs)r r5 rN r� r r� rQ r� rT r< rR r� r� r� )rv r� Z mountoutputr� Zregexr� r� r� r� r� rW r r� r� r� r� ZdevmZ match_devpthr6 r6 r7 � parse_mount^
sl �
��
�� r� c C sT dt �� }t j�|�r2t|��� }t| |||�S t j�d�rFt| �S t| |�S d S )Nz/proc/%s/mountinfor� ) r2 �getpidrv r� r� rN r� r� r� )rv r� r� Zmountinfo_pathr r6 r6 r7 �get_mount_info�
s r� )�optr^ c C s t | dd��^ }}||�d�v S )NT�r� rJ �r� r5 )rv r� r( Zmnt_optsr6 r6 r7 �
has_mount_opt�
s r� c C s( | � dd�}t|�dkr |S | d fS d S )Nr- r� )�rsplitrQ )Zdottedr� r6 r6 r7 �expand_dotted_devname�
s r� c C s� |d u r g }|d u rg }g }i }|| D ]F}z t | | | dd�||< W q( tyl ||v rh|�|� Y q(0 q(t|�r�tdjd�|�d���|S )NF�r� z Missing required files: {files}rJ )�files)r� r� r� rQ rd rE rT )rL �requiredZoptional�delimrZ r� rs r6 r6 r7 �pathprefix2dict�
s" �r� �
/proc/meminfoc C s� ddddd�}dddd �}i }t | ��� D ]t}z|�� \}}}W n" tyb |�� \}}d
}Y n0 |r~t|�|| ||< q*||v r*t|�|| ||| < q*|S ) Nr� � r� � @)ZkBZmB�BZgB�total�freeZ available)z MemTotal:zMemFree:z
MemAvailable:r� )r� rN r5 rd r1 ) Zmeminfo�raw�mpliersZkmapr� rW r� rG Zunitr6 r6 r7 � read_meminfo s" �
r� c
C s� | }| � d�r | dd� } n| � d�r2| dd� } dddd d
d �}| }d}|D ]$}| � |�rN|}| d t|� � }qNz t|�}W n2 ty� } ztd
| �|�W Y d}~n
d}~0 0 |d k r�td| ��t||| �S )a} Convert human string or integer to size in bytes
In the original implementation, SI prefixes parse to IEC values
(1KB=1024B). Later, support for parsing IEC prefixes was added,
also parsing to IEC values (1KiB=1024B). To maintain backwards
compatibility for the long-used implementation, no fix is provided for SI
prefixes (to make 1KB=1000B may now violate user expectations).
Future prospective callers of this function should consider implementing a
new function with more standard expectations (1KB=1000B and 1KiB=1024B)
Examples:
10M => 10485760
10MB => 10485760
10MiB => 10485760
ZiBN���r� r� r� r� r� r� l )r� �K�M�G�Tr z'%s' is not valid input.z'%s': cannot be negative)rq rQ �floatrd r1 )rD Zsize_inr� ZnumZmplierr r� r6 r6 r7 � human2bytes s&
$ r� c C s< | du rt �� d } | dkp6| d dko6| dd� dk}|S )z$Return True if platform is x86-basedNr� Zx86_64r r� r. Z86)r2 r3 )Z
uname_archZx86_arch_matchr6 r6 r7 �is_x86D s �r� c C s
t �| �S r� )�email�message_from_string)r� r6 r6 r7 r� N s r� c C sx t j ddgdd�} t� }| j�� D ]P}z|�d d�\}}}W n tyR Y q"Y n0 |�d�r"|�t� dd|�� q"|S ) Nz
dpkg-queryz--listTr9 r. )�hi�iiz:.*r� )
r r� r; rN r5 rd r� r� r� �sub)r= Z pkgs_instrW �stater� r( r6 r6 r7 �get_installed_packagesR s
r� c
C s� d} z.t t| dd��}|�dd��� dkr0W dS W n2 tyd } zt�d| |� W Y d }~n
d }~0 0 t� }d|v rxdS td dd�}d|�� v r�dS tj �
d
�r�dS d S ) Nr Tr� r� r� z ubuntu-corez!Unexpected error loading '%s': %sz
snap_core=z /etc/system-image/channel.iniz/etc/system-image/config.d/F) r r� r r� rd rR rS r� r2 rv r[ )ZorpathZorinfor� r rN r6 r6 r7 �system_is_snappya s
$ r� c C s� d }| � � D ] }|�d�r |dd � } q,q |d u r8d S |�d�rF|S |�d�rdd|td�d � S |�d�r�d|td�d � �� S |�d�r�d |td�d � �� }tj�|�r�|S t|�}|r�|d
S |S d| S ) Nzroot=rK r� r� z/dev/disk/by-label/zUUID=z/dev/disk/by-uuid/z PARTUUID=z/dev/disk/by-partuuid/r )r5 r� rQ r� r2 rv r� r� )r r r Z
disks_path�resultsr6 r6 r7 �rootdev_from_cmdliney s.
� r� c C sF dd� }i }|| �D ],}|� dd�\}}|s0|}|s8|r|||< q|S )z�Given shell like syntax (key=value\nkey2=value2\n) in content
return the data in dictionary form. If 'add_empty' is True
then add entries in to the returned dictionary for 'VAR='
variables. Set their value to empty_val.c S s t j| dd�S )NT)Zcomments)�shlexr5 )r] r6 r6 r7 � _shlex_split� s z(load_shell_content.<locals>._shlex_splitr� r� )r5 )rN Z add_emptyZ empty_valr� rG rW r� rG r6 r6 r7 r � s
r � �?c C s� t | �}d}|t dd� |D ��8 }t|�dkrBt�d||| � g S |dkrZt�d||| � || |krhq|t�|� ||7 }q t�d|||� |S )Nr c S s g | ]}t j�|�r|�qS r6 r� rr r6 r6 r7 rH � rI z"wait_for_files.<locals>.<listcomp>z)%sAll files appeared after %s seconds: %sz6%sWaiting up to %s seconds for the following files: %sz*%sStill missing files after %s seconds: %s)r� rQ rR r� r9 �sleep)�flistZmaxwaitZnaplenZlog_preZneedZwaitedr6 r6 r7 �wait_for_files� s4 ��
�r� c C s dd� }| j d|g td� dS )z-Helper to wait on completion of snap seeding.c S s* t �d�st�d� d S t � g d�� d S )N�snapz+Skipping snap wait, no snap command present)r� �waitr� z seed.loaded)r rv rR r� r6 r6 r6 r7 r& � s
z&wait_for_snap_seeded.<locals>.callbackz snap-seeded)ZfreqN)�runr& )r� r& r6 r6 r7 �wait_for_snap_seeded� s r� c C s& t | dd�}|d �d�}|d dkS )z1Check whether the given mount point is mounted rwTr� r� rJ r �rwr� )r� r� Z
mount_optsr6 r6 r7 �mount_is_read_write� s r� c C sX t �d�sdS ddg}| r:tj�| �r*dS |�d| g� |rN|�d| g� t � |�S )zAInvoke udevadm settle with optional exists and timeout parametersZudevadmNZsettlez--exit-if-exists=%sz --timeout=%s)r rv r2 rv r� r� )r� rN Z
settle_cmdr6 r6 r7 �udevadm_settle� s
r� r' )r` r^ c
C s� dg i}z t �t| �d�dd��}W nX tyH t�d| �d�� Y nF t jy| } z t�dt j
|� W Y d }~nd }~0 0 d|vr�g |d< |S )NZscopeszhotplug.enabledFr� zFile not found: %sz>Ignoring contents of %s because it is not decodable. Error: %s) r r
r� r_ r� rR r� ZJSONDecodeErrorrS r ZHOTPLUG_ENABLED_FILE)r` rN r� r6 r6 r7 �read_hotplug_enabled_file� s � �r� c c s
dV dS )ztContext manager that does nothing.
Note: In python-3.7+, this can be substituted by contextlib.nullcontext
Nr6 r6 r6 r6 r7 � nullcontext s r� )r\ )r\ )N)N)N)r� N)N)TT)F)N)r )N)N)N)NN)F)r� r� rK )N)r� r� rK rj )N)N)F)r� )Nr� NFN)Nr� NFN)Nr� NFN)Nr� NFN)Nr� NFN)NF)r� N)NN)NN)N)r )NNN)NNNT)F)rP )rP r+ F)r� rk )T)NN)r\ r� )F)r� F)N)FN)r� r� )NN)�rk �
contextlibr7 r* r� r� r! r� r� r� r Zloggingr2 Zos.pathr� r� r� r� r� rG r� r� r� r3 r0 r9 �base64r � collectionsr r r r r� r � functoolsr �pathlibr �typesr
�typingr r r
r r r r r r r r r r Zurllibr rc Z cloudinitr r r r r r r r r! r" r# Zcloudinit.log.log_utilr$ Zcloudinit.settingsr% r& Zcloudinit.helpersr' r� Z getLoggerr� rR �sepr� r� r� r� r� r� r8 r> rB rb rf rd rh Ztimedrm rq rr r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� rV r� r� r� r� r� r� r� r� r� r� r r r r� r r r r r> r@ r rF rJ rZ ra r� rS r{ r} r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� �PathLiker1 r� r� r� r� r� r� r� r� r r� r� r� r� r� r r r
r r r r) r, r. r4 r6 r= rI rJ rO rR rS r rU rX r^ rb rg rM rj rn rr ru rw ry rz r{ r� r� r} r� r� r� r� r� r� r� r� r� r� r� r� rv r� r� r� r� r� r� r� r� r r� r� r� r� r� r� r6 r6 r6 r7 �<module> s
< 4
�
/
4
(
=:
T8
!( 7 / #� + (3�
�
�
�
�
@
� � � �
� b.
- �
^
*���
���= %
- �
� N
W' +