Skip to content

Add short_metrics#4148

Merged
JackCaoG merged 5 commits into
masterfrom
short_metrics_report
Nov 3, 2022
Merged

Add short_metrics#4148
JackCaoG merged 5 commits into
masterfrom
short_metrics_report

Conversation

@JackCaoG

@JackCaoG JackCaoG commented Nov 2, 2022

Copy link
Copy Markdown
Collaborator

This pr added an api short_metrics_report which by defualt will emit a very short list of relevant metrics. User can also customize which metrics they want to see. IMO full metrics is way too detail in most time.

We will always print out the aten:: counters since they indicate fallback and are important to users.

example output:

Metric: CompileTime
  TotalSamples: 2
  Accumulator: 059ms522.421us
  ValueRate: 01s330ms003.908us / second
  Rate: 45.4528 / second
  Percentiles: 1%=026ms399.367us; 5%=026ms399.367us; 10%=026ms399.367us; 20%=026ms399.367us; 50%=032ms123.054us; 80%=032ms123.054us; 90%=032ms123.054us; 95%=032ms123.054us; 99%=032ms123.054us
Metric: ExecuteTime
  TotalSamples: 3
  Accumulator: 017ms966.371us
  ValueRate: 322ms039.563us / second
  Rate: 56.9432 / second
  Percentiles: 1%=001ms307.637us; 5%=001ms307.637us; 10%=001ms307.637us; 20%=001ms307.637us; 50%=008ms736.607us; 80%=008ms922.127us; 90%=008ms922.127us; 95%=008ms922.127us; 99%=008ms922.127us
Metric: TransferToServerTime
  TotalSamples: 1
  Accumulator: 002ms847.252us
  Percentiles: 1%=002ms847.252us; 5%=002ms847.252us; 10%=002ms847.252us; 20%=002ms847.252us; 50%=002ms847.252us; 80%=002ms847.252us; 90%=002ms847.252us; 95%=002ms847.252us; 99%=002ms847.252us
Metric: TransferFromServerTime
  TotalSamples: 3
  Accumulator: 013ms067.510us
  ValueRate: 279ms381.785us / second
  Rate: 64.1396 / second
  Percentiles: 1%=850.899us; 5%=850.899us; 10%=850.899us; 20%=850.899us; 50%=005ms494.984us; 80%=007ms721.627us; 90%=007ms721.627us; 95%=007ms721.627us; 99%=007ms721.627us
Counter: CachedCompile
  Value: 1
Counter: MarkStep
  Value: 2
Counter: aten::_local_scalar_dense
  Value: 1

@JackCaoG JackCaoG changed the title [WIP]Add short_metrics Add short_metrics Nov 2, 2022
@JackCaoG JackCaoG force-pushed the short_metrics_report branch from ee594e8 to c70be32 Compare November 2, 2022 21:16
Comment thread torch_xla/debug/metrics.py Outdated
Comment thread test/test_metrics.py Outdated

@will-cromar will-cromar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise LGTM

@JackCaoG JackCaoG merged commit 1f37069 into master Nov 3, 2022
alanwaketan added a commit to pytorch/pytorch that referenced this pull request Nov 17, 2022
Summary:
This pull request upstreams pytorch/xla#4148.

Test Plan:
xla CI.
pytorchmergebot pushed a commit to pytorch/pytorch that referenced this pull request Nov 18, 2022
Summary:
This pull request upstreams pytorch/xla#4148.

Test Plan:
xla CI.

Pull Request resolved: #89186
Approved by: https://github.com/JackCaoG
kulinseth pushed a commit to kulinseth/pytorch that referenced this pull request Dec 10, 2022
Summary:
This pull request upstreams pytorch/xla#4148.

Test Plan:
xla CI.

Pull Request resolved: pytorch#89186
Approved by: https://github.com/JackCaoG
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 25, 2026
Summary:
This pull request upstreams pytorch/xla#4148.

Test Plan:
xla CI.

Pull Request resolved: pytorch#89186
Approved by: https://github.com/JackCaoG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants