Discussion:
Glad to see Raxco OpenVMS performance suite still around
(too old to reply)
u***@gmail.com
2017-05-22 12:27:42 UTC
Permalink
Raw Message
ran it for years, great product.
Perfectdisk, perfectcache and perfect tune where
great apps.


Performance Suite

Most OpenVMS systems experience: disk fragmentation, random file placement, I/O bottlenecks, ineffective sharing of memory and CPU among processes, and a lack of monitoring and reporting tools to gauge the scope of these problems.

The Performance Suite integrates high-performance system resource management solutions that boost productivity and the value of your resources.

Performance Suite Key Benefits

Improves I/O performance by utilizing unused main memory to store data, reducing I/Os to physical disk.
Eliminates file fragmentation with a single pass on the disk, open-file defragmentation included!
Intelligently positions files on the disk to minimize average seek times and delay re-fragmentation
Tunes the system automatically in response to changing workloads and interactive usage.
Eliminates RMS index file fragmentation on-line and tunes FDL settings.
Jan-Erik Soderholm
2017-05-22 12:39:46 UTC
Permalink
Raw Message
Post by u***@gmail.com
ran it for years, great product.
Perfectdisk, perfectcache and perfect tune where
great apps.
Performance Suite
Most OpenVMS systems experience: disk fragmentation, random file placement, I/O bottlenecks, ineffective sharing of memory and CPU among processes, and a lack of monitoring and reporting tools to gauge the scope of these problems.
The Performance Suite integrates high-performance system resource management solutions that boost productivity and the value of your resources.
Performance Suite Key Benefits
Improves I/O performance by utilizing unused main memory to store data, reducing I/Os to physical disk.
Eliminates file fragmentation with a single pass on the disk, open-file defragmentation included!
Intelligently positions files on the disk to minimize average seek times and delay re-fragmentation
Tunes the system automatically in response to changing workloads and interactive usage.
Eliminates RMS index file fragmentation on-line and tunes FDL settings.
I see very litte use for these products using todays HW and SAN storage.
Stephen Hoffman
2017-05-22 14:57:25 UTC
Permalink
Raw Message
Post by u***@gmail.com
ran it for years, great product.
Perfectdisk, perfectcache and perfect tune where great apps.
So let us look at this, from a somewhat more recent perspective...

We have some unsigned kernel code with full security access, optimizing
a very old and problematic and inherently slow I/O path (and one with
more than a little glue code), and which is certainly going to be
useful for some folks that are still on that I/O path. All good, for
folks that are still using that approach, but not exactly something I'd
expect folks to really want to choose going into 2021 or 2027.

What do I mean by this? Signed code and full disk encryption and some
replacement for RMS are usually going to be preferred by app developers
going forward, whether it's some combination of an integrated
relational database or a distributed object store or otherwise, and
with an I/O path that's a little less tedious to deal with and to code
and to access than RMS, and preferably a path that provides a mechanism
for online backups and whatever ongoing maintenance is required and
other such other features. Various of these are increasingly available
on other platforms, whether it's sandboxed and signed code — consider
that caching products share some similarities with anti-malware tools
in terms of the required level of system trust and the depth at which
they're integrated into the system and that they're all attack targets
and they have access to potentially very sensitive data. These are
all areas that VSI knows about and will be working on going forward,
though they've work on Kittson and the x86-64 port and more 64-bit
support ahead of that work.

Computing five or ten years out — where folks might choose to look at
and start basing their apps and partnerships on OpenVMS — is certainly
going to build on some of what OpenVMS has, but there's more than a
little work to make it interesting to wholly new folks and wholly new
apps.

What can be easily overlooked is that OpenVMS is not going to be
competing on mail servers, nor on RMS-based file I/O nor with tools to
overcome platform limitations with same. Not in the next five or ten
years. OpenVMS has to compete on what partners can do with it, and
that comes down to how much revenue can be accrued by the partners, how
much risk arises, pricing differentials, and that OpenVMS is presently
pretty close to an embedded platform in terms of how it's packaged and
deployed and updated and managed.

Products and features that are already available aren't going to be a
huge draw because, well, they haven't been. Look where OpenVMS is now
in the market. VSI is the only reason there's as much activity as
there is, and they're working diligently to establish and grow a
revenue stream, and to build up a partner network. They're not big
enough to keep up with what Microsoft and Linux and other platforms can
do, so the folks at VSI are focusing on what they can do to keep the
installed base happy, and then at what they need to start to do to grow
the base, and at what can draw in and keep the partners — at the next
couple of discussions and implementations and projects and partnerships
beyond the simplistic "revenues" mention, that is. That's also 2021
to 2027 before the VSI folks and third-parties start to have those
pieces in place, at the earliest.

The add-on Raxco tools and the add-on mail servers and the rest aren't
really aimed at attracting new folks, and there's more than a little
work around to get OpenVMS into a more competitive position in the
market outside the installed base, so the question becomes what areas
to focus on, and what areas to ignore, and what areas to market on.
VSI has chosen security for marketing — as have you and some other
folks here — though that comes with some discussions around getting
TLSv1.3 and later (and maybe frameworks to make that task and
networking easier) and integrated certificate stores and removing and
deprecating insecure transports and insecure or outdated
implementations, and a variety of discussions already posted. Work.
Lots of it.

Competing head-on with Windows Server and Exchange Server, or with
Postfix, Dovecot and the open source tools, isn't something I see as
being viable — if you do, certainly start offering that service to
folks, of course – but there are areas where an embedded platform can
compete. But it'll take more than a little time and effort, and
pointing to existing tools isn't (hasn't) going to do that, and hasn't
done that. Many folks reasonably don't want to manage the resulting
infrastructure, and don't have and don't want to acquire those too.
Those that do want more automation around those tasks, as we need to
install updates far more quickly and efficiently, and we need to reduce
the effort involved when applications are breached, or when
applications just stomp on each other — and system- and kernel-mode
extensions are really good at stomping on other things, from direct
experience.

You're seemingly very focused on what was, on the past, on how things
were once done, on history, and on what's familiar and perceived as
safe. At what competing products were, too. Fun times for some,
expensive and bespoke times for most, and an era of isolated servers
and partitioned networks and that often didn't need much in the way of
security or security knowledge, and we're just not and never will be
headed back to those days. Look around. Learn. Actually try some
of the competing products. Potential future customers of OpenVMS or
any platform certainly perform those comparisons now, and and will
continue to do that, too.

FWIW...

FWIW (1) and since there's been some "it's in draft" comments posted
about TLSv1.3, well, sure. The draft is has already been deployed in
some security-relevant products, and more are coming online. Stuff in
draft status is how a vendor has the features and capabilities
available when the release is available. The same holds for
announcing products and product marketing these days, but I digress.
Products that market on security usually want to select and track
appropriate security features, and TLS is one of the key security
features on any platform and in the many apps where IPSec doesn't and
can't fit. We're increasingly all dealing with mobile client devices
and far larger and far less trusted networks, and with all that
entails.
https://blog.cloudflare.com/tls-1-3-explained-by-the-cloudflare-crypto-team-at-33c3/


FWIW (2), we've come a long way from the environments that many of
OpenVMS users and developers started out in — massively different from
when I started, certainly — and requirements and expectations are
moving much more quickly, and products and vendors can either keep up
or the applications and environments transition into legacy status and
eventually set aside and maintained until replacement. Not that there
isn't a market for legacy products, but that path inevitably drifts
into irrelevance and oblivion. Something I'd prefer not to see happen
with OpenVMS. Competing products have gotten massively better, and
replacements and alternatives to tentpole OpenVMS features are now
commonplace — I just bumped into VMware's Herat, for instance.
https://github.com/vmware/haret/blob/master/docs/why.md That 65% of
data breaches are due to humans was a bit of a surprise, too, that per
the current Verizon DBIR.

Times and risks and expectations and businesses and the climate and the
competitive landscape all change, and each with significant effects on
existing products and new product development. Learn from the past,
but don't rest on the past. Always look forward.
--
Pure Personal Opinion | HoffmanLabs LLC
Kerry Main
2017-05-22 16:01:40 UTC
Permalink
Raw Message
-----Original Message-----
Stephen Hoffman via Info-vax
Sent: May 22, 2017 10:57 AM
Subject: Re: [Info-vax] Glad to see Raxco OpenVMS performance suite
still around
Post by u***@gmail.com
ran it for years, great product.
Perfectdisk, perfectcache and perfect tune where great apps.
So let us look at this, from a somewhat more recent perspective...
We have some unsigned kernel code with full security access, optimizing
a very old and problematic and inherently slow I/O path (and one with
more than a little glue code), and which is certainly going to be
useful for some folks that are still on that I/O path. All good, for
folks that are still using that approach, but not exactly something I'd
expect folks to really want to choose going into 2021 or 2027.
[snip]
Competing head-on with Windows Server and Exchange Server, or with
Postfix, Dovecot and the open source tools, isn't something I see as
being viable — if you do, certainly start offering that service to
folks, of course – but there are areas where an embedded platform can
compete. But it'll take more than a little time and effort, and
pointing to existing tools isn't (hasn't) going to do that, and hasn't
done that. Many folks reasonably don't want to manage the resulting
infrastructure, and don't have and don't want to acquire those too.
Those that do want more automation around those tasks, as we need to
install updates far more quickly and efficiently, and we need to reduce
the effort involved when applications are breached, or when
applications just stomp on each other — and system- and kernel-mode
extensions are really good at stomping on other things, from direct
experience.
You're seemingly very focused on what was, on the past, on how things
were once done, on history, and on what's familiar and perceived as
safe. At what competing products were, too. Fun times for some,
expensive and bespoke times for most, and an era of isolated servers
and partitioned networks and that often didn't need much in the way of
security or security knowledge, and we're just not and never will be
headed back to those days. Look around. Learn. Actually try some
of the competing products. Potential future customers of OpenVMS or
any platform certainly perform those comparisons now, and and will
continue to do that, too.
You are also guilty somewhat of promoting the distributed ways of the past i.e. with Windows/Linux / UNIX [platforms, the App developer needs to put in their Application code: (think of the additional complexity this requires)
- data consistency and availability (replication associated with shared nothing, stateless models)
- node mgmt. in shared nothing models e.g. when nodes are added, deleted, put in maint mode), the App code needs to deal with this
- inter-node communications wrt to write updates
- disaster recovery

Leaving aside some of the catch-up issues currently faced, with the OpenVMS model, these issues are addressed at the OS level and allows the developers to focus more on their core focus - application quality and optimizations.

Reference: " Making the Case for Building Scalable Stateful Services in the Modern Era"
<http://highscalability.com/blog/2015/10/12/making-the-case-for-building-scalable-stateful-services-in-t.html>

The question I would ask is "based on this article, is a centralized, high availability, tightly coupled shared disk compute OpenVMS model better suited to address next generation compute solutions or todays highly distributed model of loosely coupled, shared nothing stateless models associated with commodity OS's?"

You provided this Twitter Developer video awhile back - listen to all of the issues that current distributed developers need to address. Then recognize that most of the issues she raises are handled at the OS/cluster level in the OpenVMS model.

"The Stateless Service design principle has become ubiquitous in the tech industry for creating horizontally scalable services. However our applications do have state, we just have moved all of it to caches and databases. Today as applications are becoming more data intensive and request latencies are expected to be incredibly low, we’d like the benefits of stateful services, like data locality and sticky consistency. In this talk I will address the benefits of stateful services, how to build them so that they scale, and discuss projects from Halo and Twitter of highly distributed and scalable services that implement these techniques successfully."

Of course, it depends on the App requirements, but imho, there are a lot of companies that are going to want to significantly reduce overall software costs, reduce developer complexities and move to much more centralized models.

The future offers all sorts of new application opportunities on X86-64 / post x86-64 OpenVMS platforms.

-
FWIW...
FWIW (1) and since there's been some "it's in draft" comments posted
about TLSv1.3, well, sure. The draft is has already been deployed in
some security-relevant products, and more are coming online.
And drafts change (sometimes in major ways) before final release, so stating OpenVMS is not current because various products or core features are not XYZ draft compliant is a bit weak argument.

Vendors and more important, Customers on other platforms avoid implementation of draft standards as well for the same reasons.


Regards,

Kerry Main
Kerry dot main at starkgaming dot com
V***@SendSpamHere.ORG
2017-05-22 15:17:53 UTC
Permalink
Raw Message
Post by u***@gmail.com
ran it for years, great product.
Perfectdisk, perfectcache and perfect tune where
great apps.
Performance Suite
Most OpenVMS systems experience: disk fragmentation, random file placement, I/O bottlenecks, ineffective sharing of memory and CPU among processes, and a lack of monitoring and reporting tools to gauge the scope of these problems.
The Performance Suite integrates high-performance system resource management solutions that boost productivity and the value of your resources.
Performance Suite Key Benefits
Improves I/O performance by utilizing unused main memory to store data, reducing I/Os to physical disk.
Eliminates file fragmentation with a single pass on the disk, open-file defragmentation included!
Intelligently positions files on the disk to minimize average seek times and delay re-fragmentation
Tunes the system automatically in response to changing workloads and interactive usage.
Eliminates RMS index file fragmentation on-line and tunes FDL settings.
"Eliminates RMS index file fragmentation on-line and tunes FDL settings." ?????

That sounds like PerfectFile. I wasn't aware that that had been bundled with
Raxco Performance Suite products.
--
VAXman- A Bored Certified VMS Kernel Mode Hacker VAXman(at)TMESIS(dot)ORG

I speak to machines with the voice of humanity.
David Froble
2017-05-22 17:42:50 UTC
Permalink
Raw Message
Post by u***@gmail.com
ran it for years, great product.
Perfectdisk, perfectcache and perfect tune where
great apps.
Performance Suite
Most OpenVMS systems experience: disk fragmentation, random file placement, I/O bottlenecks, ineffective sharing of memory and CPU among processes, and a lack of monitoring and reporting tools to gauge the scope of these problems.
The Performance Suite integrates high-performance system resource management solutions that boost productivity and the value of your resources.
Performance Suite Key Benefits
Improves I/O performance by utilizing unused main memory to store data, reducing I/Os to physical disk.
Eliminates file fragmentation with a single pass on the disk, open-file defragmentation included!
Intelligently positions files on the disk to minimize average seek times and delay re-fragmentation
Tunes the system automatically in response to changing workloads and interactive usage.
Eliminates RMS index file fragmentation on-line and tunes FDL settings.
There was a time when some of these uitlities were helpful. If you're still
living in that time, perhaps they can still be helpful.

But, we're already in the time of SSDs and such. Not only do such not need
defragged, files placed, and such, but such activity can be detrimental.

Performance? Process shrinks and gobs of memory seem to have solved many such
problems.

Yesterday's tools are not always applicable or helpful in today's and tomorrow's
environments.
Kerry Main
2017-05-22 19:09:00 UTC
Permalink
Raw Message
-----Original Message-----
David Froble via Info-vax
Sent: May 22, 2017 1:43 PM
Subject: Re: [Info-vax] Glad to see Raxco OpenVMS performance suite
still around
Post by u***@gmail.com
ran it for years, great product.
Perfectdisk, perfectcache and perfect tune where
great apps.
Performance Suite
Most OpenVMS systems experience: disk fragmentation, random file
placement, I/O bottlenecks, ineffective sharing of memory and CPU
among processes, and a lack of monitoring and reporting tools to gauge
the scope of these problems.
Post by u***@gmail.com
The Performance Suite integrates high-performance system resource
management solutions that boost productivity and the value of your
resources.
Post by u***@gmail.com
Performance Suite Key Benefits
Improves I/O performance by utilizing unused main memory to store
data, reducing I/Os to physical disk.
Post by u***@gmail.com
Eliminates file fragmentation with a single pass on the disk, open-file
defragmentation included!
Post by u***@gmail.com
Intelligently positions files on the disk to minimize average seek
times and delay re-fragmentation
Post by u***@gmail.com
Tunes the system automatically in response to changing workloads
and interactive usage.
Post by u***@gmail.com
Eliminates RMS index file fragmentation on-line and tunes FDL
settings.
There was a time when some of these uitlities were helpful. If you're still
living in that time, perhaps they can still be helpful.
But, we're already in the time of SSDs and such. Not only do such not need
defragged, files placed, and such, but such activity can be detrimental.
Performance? Process shrinks and gobs of memory seem to have solved many such
problems.
Yesterday's tools are not always applicable or helpful in today's and tomorrow's
environments.
Agree 100%.

The technology is changing so quickly that traditional system management activities are going to be challenged.

Single 7200rpm sata disk drives at 10TB size are now available for less than $500 CAD.

15TB SSD's are available today, with 60TB SSD's expected soon (ok, likely at prices only available for large enterprises with critical Apps - at least for next 1-2 years)
< https://arstechnica.com/gadgets/2016/08/seagate-unveils-60tb-ssd-the-worlds-largest-hard-drive/>

TB's of non-volatile memory expected later this year/early next year

Systems management impact questions:

- how does this impact what is shipped and installed when the entire OS and all of its options take up under 10-20GB?

- how does this impact system dump files that were always a pain as they needed to be copied off the system disk to save space?

- how does one tune process quotas, dump files and system parameters for a TB memory system?

I am sure these technology changes are going to raise all sorts of questions that will impact future vendor and sysadmin practices.

Note - I have a 2TB Seagate HW mirrored disk (unsupported LSI controller) on my V8.4-2L1 system and I keep 10+ system image backup copies and it is not even 30% full.

Times are a changing .. fast!

😊


Regards,

Kerry Main
Kerry dot main at starkgaming dot com







_______________________________________________
Info-vax mailing list
http://rbnsn.com/mailman/listinfo/info-vax_rbnsn.com
u***@gmail.com
2017-05-22 23:58:17 UTC
Permalink
Raw Message
Post by u***@gmail.com
ran it for years, great product.
Perfectdisk, perfectcache and perfect tune where
great apps.
Performance Suite
Most OpenVMS systems experience: disk fragmentation, random file placement, I/O bottlenecks, ineffective sharing of memory and CPU among processes, and a lack of monitoring and reporting tools to gauge the scope of these problems.
The Performance Suite integrates high-performance system resource management solutions that boost productivity and the value of your resources.
Performance Suite Key Benefits
Improves I/O performance by utilizing unused main memory to store data, reducing I/Os to physical disk.
Eliminates file fragmentation with a single pass on the disk, open-file defragmentation included!
Intelligently positions files on the disk to minimize average seek times and delay re-fragmentation
Tunes the system automatically in response to changing workloads and interactive usage.
Eliminates RMS index file fragmentation on-line and tunes FDL settings.
so no one in the future does not need to tune OpenVMS
anymore? And raxco can certainly modify the package to
work with the enhancements vms makes.
abrsvc
2017-05-23 00:19:12 UTC
Permalink
Raw Message
A few thoughts...

I don't think that tuning will go away, however, the focus will change.
As an example, the use of PerfectCache will likely not help much given the current technology of OpenVMS caching as well as memory sizes and disk performance. With hard drives being so slow compared to memory speeds, caching disk blocks was a significant gain in performance. With more modern drives being faster, the difference in performance is less than it was. While there is still a benefit to caching blocks, even OpenVMS now does a better job without the need for a third party software package. Couple that with the use of SSD drives which effectively eliminate the slow "seek times" usually associated with disk transfers, "caching" becomes less relevant.

The same could be said for defragging disk drives. Yes, fragmented drives will incur more overhead, but that overhead is minimal when using either SSD drives or sans. De-fragmented files can get transferred more quickly whether on a standard disk, SSD or SAN, but again the difference is much less when using an SSD or SAN.

Back to tuning...

Yes tuning will still be a skill required for complete and proper systems performance management. But, the focus now will most likely be more effective use of large memory systems rather than trading memory for disk performance.

Dan
Stephen Hoffman
2017-05-23 15:10:57 UTC
Permalink
Raw Message
so no one in the future does not need to tune OpenVMS anymore?
I certainly hope that such are no longer necessary, save for all but
rare cases.

We're certainly very far along on that path already. Many apps and
many systems had to be tuned back in the VAX era. VAX wasn't very
fast and memory was very constrained. Fewer apps needed extra tuning
help on Alpha, though there were and are some. Fewer still need this
on on Integrity, and that trend will continue with more capacious
storage and memory and far faster I/O.

We are certainly missing capabilities in OpenVMS to better automate
this and to better deal with longstanding morasses such as AUTOGEN,
too. Of taking what T4 and similar provided for performance and trend
analysis, and automating that processing, too. But the need for this
is less than it once was, and the cost of manual tuning is much higher.

Again, why would anybody new (and sane) want go through the mess that
we sometimes have to deal with now? AUTOGEN and MODPARAMS and the
whole process quota system is seriously primitive. There's not even
any way to automatically determine the expected process or system quota
requirements for an app under development, or — if security is a goal
for the operating system — maybe even where the developer should insert
a pledge call, and then there's the utter SS$_EXQUOTA idiocy —
compatibility with stupidity — inexplicably continues to exist.

Why? Why is any of this classic OpenVMS approach an advantage? Why
would a new user want to deal with this? Why isn't OpenVMS helping
developers and system administrators deal with this, or automating it?
Why not have a RUN /ANALYZE or RUN /COLLECT that acquires the data,
builds up the prerequisite parameter settings and the settings and uses
that to build a bundle integrating this whole area with the
as-yet-unavailable application-signing mechanism? Much or all of the
glue code that even experienced developers often don't include in their
applications; of testing system or process setting for stability, and
the sorts of checks that litter some application startup procedures and
some application initialization code. (This also all ties into
sandboxes and into app provisioning and app deployment automation,
where tuning and capacity are of far more interest. But the tools and
the system and API isolation and the rest necessary for these sorts of
more-density tasks don't yet exist on OpenVMS. It's all manually
managing and maintaining co-resident apps, for now.)

It'd be interesting to see an ML approach to tuning an app or a system,
too. Of modernizing the existing and archaic feedback approach, and
the non-existent performance trend analysis. But that's fodder for
another discussion.

As for trends, also add in notifications of apps that are leaking, too.
The system and development instruments — tools — available on other
platforms already make these tasks easier to recognize and resolve.

Some sites do monitor their performance and their trends to the degree
they can predict when they need a capacity upgrade. (Then there's
the whole discussion of embedding these tools and automated monitoring
that enable up-selling performance or capacity or (gag) added cores,
for the folks that don't have this same depth of app and system
instrumentation, too. A potential revenue stream for VSI, down the
road. It's all about the revenues, after all.)
And raxco can certainly modify the package to work with the
enhancements vms makes.
Adding more tools and more pieces to the configuration? More
integration and more dependencies? That's not the direction I'd want
to head.

Why the do any of us want to stay with the older tuning approaches, for
that matter? Why? Why do any of us want to deal with that? Why do
new users want to buy systems that have to be tuned, defragged,
deloused and generally tended to? Sure, there are some cases where
the configurations are still as performance- or capacity-marginal as
they were back in VAX days or when the scale of the deployment warrants
the added effort — embedded designs do have those trade-offs — but for
many deployments, this effort is unnecessary and even wasteful.

Sure, process quotas can still help from blowing out the whole system,
but the vast majority of systems and apps don't bother with that.
(How many of your RUN /DETACH commands have quota lists, and how
recently have any of you looked at your specific app quota
requirements?) There are an increasing number of OpenVMS servers tat
get rebooted to restart something that tipped over, as much of an
anathema as that approach might be to many folks. Servers reboot
faster than they did, too. Yes, tuning can help when an app is very
poorly written or when the app is very close to the system capacity,
but — beyond a decreasing number of these cases — few folks care about
that, as many of the OpenVMS systems and the hardware are either
over-configured, or it's cheaper to toss some hardware at the
performance or capacity issue.

We are not in the 1990s. Expectations change. Needs change.
Hardware changes. Sometimes massively. System and application
software and management practices and tools have to change, too.

Want OpenVMS to be more widely accepted? Being far easier to deal
with, with actually-better security, and far more automated and
automate-able management, is one way. Archaic, manual, and arcane,
not so much.
--
Pure Personal Opinion | HoffmanLabs LLC
Kerry Main
2017-05-25 02:44:33 UTC
Permalink
Raw Message
-----Original Message-----
Stephen Hoffman via Info-vax
Sent: May 23, 2017 11:11 AM
Subject: Re: [Info-vax] Glad to see Raxco OpenVMS performance suite
still around
so no one in the future does not need to tune OpenVMS anymore?
I certainly hope that such are no longer necessary, save for all but
rare cases.
We're certainly very far along on that path already. Many apps and
many systems had to be tuned back in the VAX era. VAX wasn't very
fast and memory was very constrained. Fewer apps needed extra tuning
help on Alpha, though there were and are some. Fewer still need this
on on Integrity, and that trend will continue with more capacious
storage and memory and far faster I/O.
We are certainly missing capabilities in OpenVMS to better automate
this and to better deal with longstanding morasses such as AUTOGEN,
too. Of taking what T4 and similar provided for performance and trend
analysis, and automating that processing, too. But the need for this
is less than it once was, and the cost of manual tuning is much higher.
Recent HW / storage technology improvements has reduced the need for capacity planning on ALL platforms, not just OpenVMS.

Most OpenVMS apps today (estimate 75%?) run just fine on Alpha and I2/I4 systems.

HW upgrades are usually for support, licensing and/or end-of-life issues.

I would wager that once porting issues were addressed, 75% of most apps today would run just fine on any current HW platform from any vendor using todays workloads.

Having stated this, workloads of the future are likely going to be much more demanding than they are today. Big data, machine learning, AI, Virtual reality, global workloads etc. are all going to demand higher performance, and availability levels.

Hence, there is a need to continue to focus on capacity planning - both semi-manual using traditional manual + autogen based processes + with increased automation. Its all of these.

A good tool is only as good as the carpenter using it. Pure automation without consideration of the business SLA's is a recipe for disaster. An automated capacity planning tool that sees a process running at 100% might decide to give it more memory and more cpu's without any consideration that the process is in a runaway state and is impacting the overall system.

A good example where I would like to see more work done in the future is the OpenVMS native class scheduler. It allows one to group processes and assign SLA's. this in turn does automated native resource mgmt. so that one process/group does not run away with the whole system.

Btw, OpenVMS does have commercial options for higher levels of automated capacity planning - complete with recommendations. Remember the "Performance and Capacity Planner" product that is now owned by CA?
< http://odl.sysworks.biz/disk$vaxdocsep951/decw$book/d3n1aaa2.p51.decw$book>

< http://bit.ly/2qlNfkE>
Again, why would anybody new (and sane) want go through the mess that
we sometimes have to deal with now? AUTOGEN and MODPARAMS
and the
whole process quota system is seriously primitive. There's not even
any way to automatically determine the expected process or system quota
requirements for an app under development, or — if security is a goal
for the operating system — maybe even where the developer should insert
a pledge call, and then there's the utter SS$_EXQUOTA idiocy —
compatibility with stupidity — inexplicably continues to exist.
So how does Windows/Linux do automated capacity planning?

Answer - they don't. if required, they would look to a commercial option from a vendor.
Why? Why is any of this classic OpenVMS approach an advantage? Why
would a new user want to deal with this? Why isn't OpenVMS helping
developers and system administrators deal with this, or automating it?
Why not have a RUN /ANALYZE or RUN /COLLECT that acquires the data,
builds up the prerequisite parameter settings and the settings and uses
that to build a bundle integrating this whole area with the
as-yet-unavailable application-signing mechanism? Much or all of the
glue code that even experienced developers often don't include in their
applications; of testing system or process setting for stability, and
the sorts of checks that litter some application startup procedures and
some application initialization code. (This also all ties into
sandboxes and into app provisioning and app deployment automation,
where tuning and capacity are of far more interest. But the tools and
the system and API isolation and the rest necessary for these sorts of
more-density tasks don't yet exist on OpenVMS. It's all manually
managing and maintaining co-resident apps, for now.)
It'd be interesting to see an ML approach to tuning an app or a system,
too. Of modernizing the existing and archaic feedback approach, and
the non-existent performance trend analysis. But that's fodder for
another discussion.
As for trends, also add in notifications of apps that are leaking, too.
The system and development instruments — tools — available on other
platforms already make these tasks easier to recognize and resolve.
Some sites do monitor their performance and their trends to the degree
they can predict when they need a capacity upgrade. (Then there's
the whole discussion of embedding these tools and automated
monitoring
that enable up-selling performance or capacity or (gag) added cores,
for the folks that don't have this same depth of app and system
instrumentation, too. A potential revenue stream for VSI, down the
road. It's all about the revenues, after all.)
And raxco can certainly modify the package to work with the
enhancements vms makes.
Adding more tools and more pieces to the configuration? More
integration and more dependencies? That's not the direction I'd want
to head.
Why the do any of us want to stay with the older tuning approaches, for
that matter? Why? Why do any of us want to deal with that? Why do
new users want to buy systems that have to be tuned, defragged,
deloused and generally tended to? Sure, there are some cases where
the configurations are still as performance- or capacity-marginal as
they were back in VAX days or when the scale of the deployment
warrants
the added effort — embedded designs do have those trade-offs — but for
many deployments, this effort is unnecessary and even wasteful.
Sure, process quotas can still help from blowing out the whole system,
but the vast majority of systems and apps don't bother with that.
(How many of your RUN /DETACH commands have quota lists, and how
recently have any of you looked at your specific app quota
requirements?) There are an increasing number of OpenVMS servers tat
get rebooted to restart something that tipped over, as much of an
anathema as that approach might be to many folks. Servers reboot
faster than they did, too. Yes, tuning can help when an app is very
poorly written or when the app is very close to the system capacity,
but — beyond a decreasing number of these cases — few folks care about
that, as many of the OpenVMS systems and the hardware are either
over-configured, or it's cheaper to toss some hardware at the
performance or capacity issue.
We are not in the 1990s. Expectations change. Needs change.
Hardware changes. Sometimes massively. System and application
software and management practices and tools have to change, too.
Want OpenVMS to be more widely accepted? Being far easier to deal
with, with actually-better security, and far more automated and
automate-able management, is one way. Archaic, manual, and arcane,
not so much.
There is a reason why all IT platform do not provide automated system and application tuning out of the box.

Answer - Its really, really tough to do.

Other platforms also struggle with this, because todays app complexity makes it very hard to automate.

In todays N-tier app world, if the "app is slow", is it the web server, or the app server or the DB server or the network or perhaps an overloaded common firewall that someone has decided to run backups through?

What you are talking about has been around for decades i.e. application and system and database instrumentation (hooks) which send alerts to enterprise management frameworks.

In the end, the answer is to:
- improve automation where possible and where it makes sense. Hint - my vote is for future OpenVMS class scheduler improvements - perhaps GUI interface?. Commodity OS's do not have native class schedulers because they typically do not run mixed business workloads on the same OS. Hence, this is a potential differentiator in the future for OpenVMS.
- continue manual capacity planning processes because when running different workloads on the same OS, resources need to take into account business priorities which may change from time to time.
- for critical workloads, purchase capacity planning tools to assist with the overall tuning

In summary, creating smart tools means nothing without a good carpenter that knows how and when to use them.


Regards,

Kerry Main
Kerry dot main at starkgaming dot com

GerMarsh
2017-05-23 08:06:49 UTC
Permalink
Raw Message
Post by u***@gmail.com
ran it for years, great product.
Perfectdisk, perfectcache and perfect tune where
great apps.
Performance Suite
Most OpenVMS systems experience: disk fragmentation, random file placement, I/O bottlenecks, ineffective sharing of memory and CPU among processes, and a lack of monitoring and reporting tools to gauge the scope of these problems.
The Performance Suite integrates high-performance system resource management solutions that boost productivity and the value of your resources.
Performance Suite Key Benefits
Improves I/O performance by utilizing unused main memory to store data, reducing I/Os to physical disk.
Eliminates file fragmentation with a single pass on the disk, open-file defragmentation included!
Intelligently positions files on the disk to minimize average seek times and delay re-fragmentation
Tunes the system automatically in response to changing workloads and interactive usage.
Eliminates RMS index file fragmentation on-line and tunes FDL settings.
Ah yes - Sifting through Rabbit 2 reports! I saw quite a few examples where some over-enthusiastic changing of system parameters resulted in much slower systems. I think the emphasis now should be on application performance rather than the tools and skills we had to use. I do have fond memories of Clayton J Prestia talking in some Raxco seminar though!

Some of the max values on MONITOR SYSTEM plots seem rather dated these days with SAN io being capable of much, much bigger rates.
Jan-Erik Soderholm
2017-05-23 09:01:05 UTC
Permalink
Raw Message
Post by GerMarsh
ran it for years, great product. Perfectdisk, perfectcache and perfect
tune where great apps.
Performance Suite
Most OpenVMS systems experience: disk fragmentation, random file
placement, I/O bottlenecks, ineffective sharing of memory and CPU
among processes, and a lack of monitoring and reporting tools to gauge
the scope of these problems.
The Performance Suite integrates high-performance system resource
management solutions that boost productivity and the value of your
resources.
Performance Suite Key Benefits
Improves I/O performance by utilizing unused main memory to store
data, reducing I/Os to physical disk. Eliminates file fragmentation
with a single pass on the disk, open-file defragmentation included!
Intelligently positions files on the disk to minimize average seek
times and delay re-fragmentation Tunes the system automatically in
response to changing workloads and interactive usage. Eliminates RMS
index file fragmentation on-line and tunes FDL settings.
Ah yes - Sifting through Rabbit 2 reports! I saw quite a few examples
where some over-enthusiastic changing of system parameters resulted in
much slower systems. I think the emphasis now should be on application
performance rather than the tools and skills we had to use. I do have
fond memories of Clayton J Prestia talking in some Raxco seminar
though!
Some of the max values on MONITOR SYSTEM plots seem rather dated these
days with SAN io being capable of much, much bigger rates.
He, doing a $DIR or $SEA on one of our SAN volumes easily makes
the CPU go to 100% on our system. And that is (only) a 2 Gb SAN.
The disk subsystem is faster then the CPU can handle...
Stephen Hoffman
2017-05-23 15:13:51 UTC
Permalink
Raw Message
Post by GerMarsh
Some of the max values on MONITOR SYSTEM plots seem rather dated these
days with SAN io being capable of much, much bigger rates.
More than a few of the default and the default-maximum parameter and
quota values, for that matter. How many of us are looking at
********** asterisk'd field overflows in some of our display output,
too?
--
Pure Personal Opinion | HoffmanLabs LLC
Craig A. Berry
2017-05-24 02:44:59 UTC
Permalink
Raw Message
Post by Stephen Hoffman
Post by GerMarsh
Some of the max values on MONITOR SYSTEM plots seem rather dated these
days with SAN io being capable of much, much bigger rates.
More than a few of the default and the default-maximum parameter and
quota values, for that matter. How many of us are looking at
********** asterisk'd field overflows in some of our display output, too?
There are somewhat fewer of those in the latest VSI release, but no, I
don't think they are done tracking them all down.
Loading...