XRootD
XrdPfc::DirState Struct Reference

#include <XrdPfcDirState.hh>

+ Inheritance diagram for XrdPfc::DirState:
+ Collaboration diagram for XrdPfc::DirState:

Public Types

typedef DsMap_t::iterator DsMap_i
 
typedef std::map< std::string, DirState > DsMap_t
 

Public Member Functions

 DirState ()
 Constructor. More...
 
 DirState (DirState *parent)
 
 DirState (DirState *parent, const std::string &dname)
 
int count_dirs_to_level (int max_depth) const
 
DirState * create_child (const std::string &dir)
 
void dump_recursively (const char *name, int max_depth) const
 
DirState * find_dir (const std::string &dir, bool create_subdirs)
 
DirState * find_path (const std::string &path, int max_depth, bool parse_as_lfn, bool create_subdirs, DirState **last_existing_dir=nullptr)
 
DirState * find_path_tok (PathTokenizer &pt, int pos, bool create_subdirs, DirState **last_existing_dir=nullptr)
 
int generate_dir_path (std::string &result)
 
DirState * get_parent ()
 
void init ()
 
bool is_empty_leaf () const
 
void reset_sshot_stats ()
 
void update_stats_and_usages (std::vector< DirState * > *reap_candidates=nullptr)
 
void upward_propagate_initial_scan_usages ()
 
- Public Member Functions inherited from XrdPfc::DirStateBase
 DirStateBase ()
 
 DirStateBase (const std::string &dname)
 

Public Attributes

int m_depth
 
DirStats m_here_stats
 
DirUsage m_here_usage
 
DirState * m_parent = nullptr
 
DirStats m_recursive_subdir_stats
 
DirUsage m_recursive_subdir_usage
 
long long m_RecursiveStBlocksFuzz = 0
 sum over the subdirs, set by propagation More...
 
bool m_scanned = false
 
DirStats m_sshot_stats
 
long long m_StBlocksFuzz = 0
 
DsMap_t m_subdirs
 
- Public Attributes inherited from XrdPfc::DirStateBase
std::string m_dir_name
 

Detailed Description

Definition at line 56 of file XrdPfcDirState.hh.

Member Typedef Documentation

◆ DsMap_i

typedef DsMap_t::iterator XrdPfc::DirState::DsMap_i

Definition at line 59 of file XrdPfcDirState.hh.

◆ DsMap_t

typedef std::map<std::string, DirState> XrdPfc::DirState::DsMap_t

Definition at line 58 of file XrdPfcDirState.hh.

Constructor & Destructor Documentation

◆ DirState() [1/3]

DirState::DirState ( )

Constructor.

Definition at line 12 of file XrdPfcDirState.cc.

12  : m_parent(0), m_depth(0)
13 {}
DirState * m_parent

Referenced by create_child().

+ Here is the caller graph for this function:

◆ DirState() [2/3]

DirState::DirState ( DirState *  parent)

Constructor

Parameters
DirStateparent directory

Definition at line 19 of file XrdPfcDirState.cc.

19  :
22 {}
static void parent()

◆ DirState() [3/3]

DirState::DirState ( DirState *  parent,
const std::string &  dname 
)

Constructor

Parameters
parentparent DirState object
dnamename of this directory only, no slashes, no extras.

Definition at line 29 of file XrdPfcDirState.cc.

29  :
30  DirStateBase(dname),
33 {}

Member Function Documentation

◆ count_dirs_to_level()

int DirState::count_dirs_to_level ( int  max_depth) const

Definition at line 218 of file XrdPfcDirState.cc.

219 {
220  int n_dirs = 1;
221  if (m_depth < max_depth)
222  {
223  for (auto & [name, ds] : m_subdirs)
224  {
225  n_dirs += ds.count_dirs_to_level(max_depth);
226  }
227  }
228  return n_dirs;
229 }

References m_depth, and m_subdirs.

Referenced by XrdPfc::ResourceMonitor::heart_beat(), and XrdPfc::ResourceMonitor::perform_purge_check().

+ Here is the caller graph for this function:

◆ create_child()

DirState * DirState::create_child ( const std::string &  dir)

Internal function called from find_dir or find_path_tok

Parameters
dirsubdir name

Definition at line 39 of file XrdPfcDirState.cc.

40 {
41  std::pair<DsMap_i, bool> ir = m_subdirs.insert(std::make_pair(dir, DirState(this, dir)));
42  return &ir.first->second;
43 }
DirState()
Constructor.

References DirState(), and m_subdirs.

Referenced by find_dir(), and find_path_tok().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ dump_recursively()

void DirState::dump_recursively ( const char *  name,
int  max_depth 
) const

Recursive print of statistics. Called if defined in pfc configuration.

Definition at line 235 of file XrdPfcDirState.cc.

236 {
237  printf("%*d %s usage_here=%lld usage_sub=%lld usage_total=%lld num_ios=%d duration=%d b_hit=%lld b_miss=%lld b_byps=%lld b_wrtn=%lld\n",
238  2 + 2 * m_depth, m_depth, name,
241  // Since the last snapshot, this directory and below.
245 
246  if (m_depth < max_depth)
247  {
248  for (auto & [name, ds] : m_subdirs)
249  {
250  ds.dump_recursively(name.c_str(), max_depth);
251  }
252  }
253 }
long long m_BytesMissed
number of bytes served from remote and cached
Definition: XrdPfcStats.hh:40
long long m_BytesBypassed
number of bytes served directly through XrdCl
Definition: XrdPfcStats.hh:41
int m_Duration
total duration of all IOs attached
Definition: XrdPfcStats.hh:38
int m_NumIos
number of IO objects attached during this access
Definition: XrdPfcStats.hh:37
long long m_BytesHit
number of bytes served from disk
Definition: XrdPfcStats.hh:39
long long m_BytesWritten
number of bytes written to disk
Definition: XrdPfcStats.hh:42
DirUsage m_recursive_subdir_usage
DirStats m_sshot_stats

References XrdPfc::Stats::m_BytesBypassed, XrdPfc::Stats::m_BytesHit, XrdPfc::Stats::m_BytesMissed, XrdPfc::Stats::m_BytesWritten, m_depth, XrdPfc::Stats::m_Duration, m_here_usage, XrdPfc::Stats::m_NumIos, m_recursive_subdir_usage, m_sshot_stats, XrdPfc::DirUsage::m_StBlocks, and m_subdirs.

Referenced by XrdPfc::DataFsState::dump_recursively().

+ Here is the caller graph for this function:

◆ find_dir()

DirState * DirState::find_dir ( const std::string &  dir,
bool  create_subdirs 
)

Non recursive function to find an entry in this directory only.

Parameters
dirsubdir name
boolcreate the subdir in this DirsStat
create_subdirsif true and the dir is not found, a new DirState child is created

Definition at line 98 of file XrdPfcDirState.cc.

100 {
101  DsMap_i i = m_subdirs.find(dir);
102 
103  if (i != m_subdirs.end())
104  return &i->second;
105 
106  if (create_subdirs)
107  return create_child(dir);
108 
109  return nullptr;
110 }
DsMap_t::iterator DsMap_i
DirState * create_child(const std::string &dir)

References create_child(), and m_subdirs.

Referenced by XrdPfc::FsTraversal::cd_down().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ find_path()

DirState * DirState::find_path ( const std::string &  path,
int  max_depth,
bool  parse_as_lfn,
bool  create_subdirs,
DirState **  last_existing_dir = nullptr 
)

Recursive function to find DirState with given absolute dir path

Parameters
pathfull path to parse
max_depthdirectory depth to which to descend (value < 0 means full descent)
parse_as_lfn
create_subdirs

Definition at line 82 of file XrdPfcDirState.cc.

84 {
85  PathTokenizer pt(path, max_depth, parse_as_lfn);
86 
87  if (last_existing_dir)
88  *last_existing_dir = this;
89 
90  return find_path_tok(pt, 0, create_subdirs, last_existing_dir);
91 }
DirState * find_path_tok(PathTokenizer &pt, int pos, bool create_subdirs, DirState **last_existing_dir=nullptr)

References find_path_tok().

Referenced by XrdPfc::DataFsState::find_dirstate_for_lfn(), and XrdPfc::ResourceMonitor::process_queues().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ find_path_tok()

DirState * DirState::find_path_tok ( PathTokenizer &  pt,
int  pos,
bool  create_subdirs,
DirState **  last_existing_dir = nullptr 
)

Internal function called from find_path

Parameters
dirsubdir name

Definition at line 49 of file XrdPfcDirState.cc.

51 {
52  if (pos == pt.get_n_dirs())
53  return this;
54 
55  DirState *ds = nullptr;
56 
57  DsMap_i i = m_subdirs.find(pt.m_dirs[pos]);
58 
59  if (i != m_subdirs.end())
60  {
61  ds = &i->second;
62  if (last_existing_dir)
63  *last_existing_dir = ds;
64  }
65  else if (create_subdirs)
66  {
67  ds = create_child(pt.m_dirs[pos]);
68  }
69 
70  if (ds)
71  return ds->find_path_tok(pt, pos + 1, create_subdirs, last_existing_dir);
72 
73  return nullptr;
74 }
std::vector< const char * > m_dirs

References create_child(), find_path_tok(), XrdPfc::PathTokenizer::get_n_dirs(), XrdPfc::PathTokenizer::m_dirs, and m_subdirs.

Referenced by find_path(), and find_path_tok().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ generate_dir_path()

int DirState::generate_dir_path ( std::string &  result)

Parent-recursively height-first append '/' and directory names to the result string. Returns total number of added characters.

Definition at line 116 of file XrdPfcDirState.cc.

117 {
118  int n_chars = 0;
119  if (m_parent) {
120  n_chars += m_parent->generate_dir_path(result) + 1 + m_dir_name.length();
121  result += '/';
122  result += m_dir_name;
123  }
124  return n_chars;
125 }
int generate_dir_path(std::string &result)

References generate_dir_path(), XrdPfc::DirStateBase::m_dir_name, and m_parent.

Referenced by generate_dir_path().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ get_parent()

DirState* XrdPfc::DirState::get_parent ( )
inline

Definition at line 97 of file XrdPfcDirState.hh.

97 { return m_parent; }

References m_parent.

Referenced by XrdPfc::FsTraversal::cd_up().

+ Here is the caller graph for this function:

◆ init()

void XrdPfc::DirState::init ( )

◆ is_empty_leaf()

bool DirState::is_empty_leaf ( ) const

No subdirs, files or open files, as of the last propagation, and no opens since. m_NDirectories can undercount, so m_subdirs is checked as well.

Definition at line 201 of file XrdPfcDirState.cc.

References m_here_stats, m_here_usage, XrdPfc::DirUsage::m_NDirectories, XrdPfc::DirUsage::m_NFiles, XrdPfc::DirStats::m_NFilesCreated, XrdPfc::DirUsage::m_NFilesOpen, XrdPfc::DirStats::m_NFilesOpened, and m_subdirs.

◆ reset_sshot_stats()

void DirState::reset_sshot_stats ( )

Definition at line 209 of file XrdPfcDirState.cc.

210 {
211  for (DsMap_i i = m_subdirs.begin(); i != m_subdirs.end(); ++i)
212  {
213  i->second.reset_sshot_stats();
214  }
216 }

References m_sshot_stats, m_subdirs, and XrdPfc::DirStats::Reset().

Referenced by XrdPfc::DataFsState::reset_sshot_stats().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ update_stats_and_usages()

void DirState::update_stats_and_usages ( std::vector< DirState * > *  reap_candidates = nullptr)

Upward propagate stats to parents, join last open/close timestamps, apply deltas / stats to usages, and reset the deltas. If reap_candidates is given, collect the empty leaves into it.

Definition at line 160 of file XrdPfcDirState.cc.

161 {
163  for (auto & [name, sub_ds] : m_subdirs)
164  {
165  sub_ds.update_stats_and_usages(reap_candidates);
166 
167  m_RecursiveStBlocksFuzz += sub_ds.m_StBlocksFuzz + sub_ds.m_RecursiveStBlocksFuzz;
168 
169  m_recursive_subdir_stats.AddUp(sub_ds.m_recursive_subdir_stats);
170  m_recursive_subdir_stats.AddUp(sub_ds.m_here_stats);
171  // nothing to do for m_here_stats.
172 
173  // Join timestamps from daugters and their sub-daugter trees.
174  m_recursive_subdir_usage.update_last_times(sub_ds.m_recursive_subdir_usage);
175  m_recursive_subdir_usage.update_last_times(sub_ds.m_here_usage);
176 
177  // The parent clears a child's deltas once absorbed; the root's are cleared by
178  // DataFsState::update_stats_and_usages().
179  sub_ds.m_here_stats.Reset();
180  sub_ds.m_recursive_subdir_stats.Reset();
181 
182  if (reap_candidates && sub_ds.is_empty_leaf())
183  reap_candidates->push_back(&sub_ds);
184  }
185 
186  // Apply current deltas to sshot_stats (and, if eventually needed, purge_stats).
189 
190  // Apply deltas / stats to usages.
193 
194  // Once we have m_report_stats and m_purge_stats, update them here.
195 }
void AddUp(const DirStats &s)
Definition: XrdPfcStats.hh:192
DirStats m_recursive_subdir_stats
long long m_RecursiveStBlocksFuzz
sum over the subdirs, set by propagation
void update_from_stats(const DirStats &s)
void update_last_times(const DirUsage &u)

References XrdPfc::DirStats::AddUp(), m_here_stats, m_here_usage, m_recursive_subdir_stats, m_recursive_subdir_usage, m_RecursiveStBlocksFuzz, m_sshot_stats, m_subdirs, XrdPfc::DirUsage::update_from_stats(), and XrdPfc::DirUsage::update_last_times().

Referenced by XrdPfc::DataFsState::update_stats_and_usages().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ upward_propagate_initial_scan_usages()

void DirState::upward_propagate_initial_scan_usages ( )

Propagate usages to parents after initial directory scan. Called from ResourceMonitor::perform_initial_scan()

Definition at line 131 of file XrdPfcDirState.cc.

132 {
133  DirUsage &here = m_here_usage;
135 
136  for (auto & [name, daughter] : m_subdirs)
137  {
138  daughter.upward_propagate_initial_scan_usages();
139 
140  DirUsage &dhere = daughter.m_here_usage;
141  DirUsage &dsubdirs = daughter.m_recursive_subdir_usage;
142 
143  here.m_NDirectories += 1;
144 
145  subdirs.m_StBlocks += dhere.m_StBlocks + dsubdirs.m_StBlocks;
146  subdirs.m_NFiles += dhere.m_NFiles + dsubdirs.m_NFiles;
147  subdirs.m_NDirectories += dhere.m_NDirectories + dsubdirs.m_NDirectories;
148 
149  // Carry the last-access times up, as update_stats_and_usages() does.
150  subdirs.update_last_times(dhere);
151  subdirs.update_last_times(dsubdirs);
152  }
153 }

References m_here_usage, XrdPfc::DirUsage::m_NDirectories, XrdPfc::DirUsage::m_NFiles, m_recursive_subdir_usage, XrdPfc::DirUsage::m_StBlocks, m_subdirs, and XrdPfc::DirUsage::update_last_times().

Referenced by XrdPfc::ResourceMonitor::perform_initial_scan().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

Member Data Documentation

◆ m_depth

◆ m_here_stats

◆ m_here_usage

◆ m_parent

DirState* XrdPfc::DirState::m_parent = nullptr

Definition at line 77 of file XrdPfcDirState.hh.

Referenced by generate_dir_path(), and get_parent().

◆ m_recursive_subdir_stats

DirStats XrdPfc::DirState::m_recursive_subdir_stats

◆ m_recursive_subdir_usage

◆ m_RecursiveStBlocksFuzz

long long XrdPfc::DirState::m_RecursiveStBlocksFuzz = 0

sum over the subdirs, set by propagation

Definition at line 75 of file XrdPfcDirState.hh.

Referenced by update_stats_and_usages().

◆ m_scanned

bool XrdPfc::DirState::m_scanned = false

Definition at line 80 of file XrdPfcDirState.hh.

Referenced by XrdPfc::ResourceMonitor::scan_dir_and_recurse().

◆ m_sshot_stats

DirStats XrdPfc::DirState::m_sshot_stats

Definition at line 69 of file XrdPfcDirState.hh.

Referenced by dump_recursively(), reset_sshot_stats(), and update_stats_and_usages().

◆ m_StBlocksFuzz

long long XrdPfc::DirState::m_StBlocksFuzz = 0

Accumulated magnitude of st_blocks corrections made at this node, never reset; a reaped child's is carried here. A diagnostic of accounting drift.

Definition at line 74 of file XrdPfcDirState.hh.

◆ m_subdirs


The documentation for this struct was generated from the following files: