Prev | Next |
Design and Build a Deck. Create a comfortable place to entertain or simply relax. This series of videos and step-by-step instructions breaks building a wood or composite deck into manageable parts. We'll show you how to create deck designs and how to build a deck from post footers to deck stairs. We'll also give you ideas to furnish. As the Director of Software Engineering for InterAction ®, I know it’s not enough to produce the best software—you need to know it’s the best, and that you’re continually striving to develop a team that can create that software in the best way possible. If you work to build a better team and provide them with clear customer feedback, they’ll be equipped to build a better product.
ClearCase provides a software build environment closely resembling that of the “traditional” make(1) program. makewas originally developed for UNIX systems, and has since been ported toother operating systems. When building software systems withClearCase-controlled data, developers can use their hosts' native make programs, their organization's home-grown shell scripts, or third-party build utilities.ClearCase includes its own build utility, clearmake, with many features not supported by other build programs:
- build auditing, with automatic detection of source dependencies, including header file dependencies
- automatic creation of permanent “bill-of-materials” documentation of the build process and its results
- sophisticated build-avoidance algorithms, guaranteeing correct results when building in a parallel development environment
- sharing of binaries among users' views, saving both time and disk storage
- parallel and distributed building, applying the resources ofmultiple processors and/or multiple hosts to builds of large softwaresystems
As described in Chapter 4, “ClearCase Views,” each view provides a complete environment for building software, including:
- a particular configuration of source versions
- a private workspace in which the user(s) can modify sourcefiles, and can use build tools to create object modules, executables,and so on
The user can (but need not) determine what other builds have takenplace in a directory, across all views. ClearCase includes tools forlisting and comparing past builds.
The key to this scheme is the fact that the development team's VOBsconstitute a globally-accessible repository for files created by builds— the same role they play for the source files that go into builds. Asharable file produced by a software build is termed a derived object (DO). Associated with each derived object is a configuration record (CR), which clearmake can use during subsequent builds to decide whether or not the DO can be reused or shared.
Figure 5-1 illustrates the ClearCase software build scheme.
Figure 5-1. Building Software with ClearCase: Isolation and Sharing
The next section describes the way in which ClearCase keeps track of the objects produced by software builds. Later, in “Build Avoidance”, we describe the mechanism that enables such objects to be shared among views.
![Software Software](http://instarama.net/wp-content/uploads/2017/08/Building-your-Software-Product.png)
- versions of elements used as build input
- view-private files used as build input — for example, the checked-out version of a file element
- files created within VOB directories during the build
Since auditing of MVFS files is completely automatic, users need notkeep track of exactly which files are being used in builds — that'sClearCase's job. For example, ClearCase determines exactly whichC-language source files referenced with #include directivesare used in a build. This eliminates the need to declare such files inthe makefile, and eliminates the need for dependency-detection tools,such as makedepend.
A build can also involve files that are not accessed through VOB directories. Such non-MVFS filesare not audited automatically, but are tracked “on request” (by beingdeclared as dependencies in a makefile). This enables auditing of buildtools that are not stored as ClearCase elements (for example, aC-language compiler), flag files in the user's home directory, and soon. Tracking information on a non-MVFS file includes its time-modifiedstamp, size, and checksum.
When it finishes executing a build script, ClearCase records theresults, including build audit information, in the form of derivedobjects and configuration records.For each new MVFS file (pathname within a VOB) created during abuild, ClearCase creates a corresponding object in the VOB database.Together, the file and the database object constitute a derived object, or DO.
All the derived objects created by execution of a build script haveequal status, even though some of them might be explicit build targets,and others might be created as “side effects” (for example, compilerlisting files). The term siblingsdescribes a group of DOs created by the same script. Siblings “traveltogether” — whenever a DO becomes shared among views, all its siblingsbecome shared, too.
A derived object is automatically assigned a unique identifier, its DO-ID; this enables users to access any derived object with an extended pathname (Figure 5-2).Figure 5-2. Extended Pathname of a Derived Object — DO-ID
A derived object's DO-ID is analogous to a version's version-ID:it is unique, globally accessible, and view-independent; itdistinguishes a particular object in the VOB database from others thatshare the same file system pathname.
As development progresses, many DOs will be created at the samepathname, by builds performed in different views, and by rebuildswithin a given view. But no name collisions occur, because each derivedobject has its own DO-ID.
For the most part, users need not be concerned with DO-IDs; they canuse a derived object's standard pathname when working in the view whereit was built (or any other view that shares it):
- To standard software — for example, linkers and debuggers — the standard pathname of a derived object (util.o) references the data file.
- To ClearCase commands, the same standard pathname references theVOB database object — for example, to access the contents of theassociated configuration record.
- A version of an element appears in a view by virtue of having been selected by a config spec rule.
- A particular derived object appears in a view as the result of a ClearCase build.
Users can examine CRs to see a wealth of information:
- Input files — Identifiers for objects used as build input: version-IDs (for versions of elements); DO-IDs(for derived objects created as subtargets); timestamps (forview-private files and for each non-MVFS file explicitly declared as adependency in the makefile)
- Output files — The pathname and DO-ID of each derived object created during the build (all the siblings)
- Build procedure — The text of the makefile's build script, including expansions of all make macros
- General data— The user who performed the build; the host used to execute the buildscript; the view in which the build took place; the date and time atwhich build script execution started
- DO listing — A user can list all the DOs created at a particular pathname, across all views:
- CR display — A user can display the contents of an individual CR:
- CR comparison — ClearCase can use CRs to compare different builds of the same target, showing differences in how the objects were built:
- Version annotation— A user can annotate some or all the versions listed in a CR (or anentire CR hierarchy) with a particular version label or attribute.
- View configuration — A user can configure a view to select exactly the versions that were used to build a particular DO, or set of DOs.
- Build management — ClearCase uses DOs and CRs to implement its build-avoidance and derived-object-sharing capabilities.
clearmake's build-avoidance algorithm does guarantee correct results. Moreover, clearmake
![List List](http://www.mbsweb.com/images/building_shape.png)
When requested to update a build target, clearmake first determines whether an existing derived object in the current view qualifies for reuse. If not, clearmake evaluates other existing derived objects that were built at the same pathname.
The process of “qualifying” a candidate DO is termed configuration lookup. It involves matching the DO's configuration record against the user's current build configuration (Figure 5-3):
- Files— The version of an element listed in the CR must match the versionselected by the user's view; any view-private files or non-MVFS fileslisted in the CR must also match.
- Build procedure — The buildscript listed in the CR must match the script that will be executed ifthe target is rebuilt. The scripts are compared with all make macrosexpanded; thus, a match occurs only if the same build options apply(for example, “compile for debugging”).Figure 5-3. Configuration Lookup
- Reuse — If the DO (and its siblings) already in the user's view match the build configuration, clearmake simply keeps them.
- Wink-in — If another, previously-built DO matches the build configuration, clearmake causes the DO and its siblings to appear in this view. This operation is termed wink-in.
- Rebuild — If configuration lookup fails to find any DO that matches the build configuration, clearmake executes the target's build script. This creates one or more new DOs, and a new CR.
In a hierarchical build, some objects are built and then used to build others. clearmakeperforms configuration lookup separately for each target. To ensure aconsistent result, it also applies this principle: if a new object iscreated, then all targets that depend on it must be rebuilt.
Some organizations, or some individual users, may want to use ClearCase build auditing without using the clearmake program. Others may want to audit development activities that do not involve makefiles at all. These users can do their work in an audited shell, a standard shell with build auditing enabled.
For example, a technical writer might produce formatted manual page files by running a shell script that invokes nroff(1). When the script is executed in a shell created by clearaudit,ClearCase creates a single configuration record, recording all thesource versions used. All MVFS files read during execution of theaudited shell are listed as inputs to the “build”. All MVFS filescreated become derived objects, associated with the singleconfiguration record.
This section discusses the physical storage of derived objects (DOs) and configuration records (CRs). As first discussed in “Derived Objects and Configuration Records”:For example, a technical writer might produce formatted manual page files by running a shell script that invokes nroff(1). When the script is executed in a shell created by clearaudit,ClearCase creates a single configuration record, recording all thesource versions used. All MVFS files read during execution of theaudited shell are listed as inputs to the “build”. All MVFS filescreated become derived objects, associated with the singleconfiguration record.
- A derived object is a compound entity, consisting of both a data file (its data container) and a corresponding VOB database object (which has a unique DO-ID).
- Each derived object has an associated configuration record.
Figure 5-4. Storage of Derived Objects and Configuration Records
Many make utilities are available in the multiple-architecture, multiple-vendor world of open systems. ClearCase's clearmake utility shares features with many of them, and has some unique features of its own.
Users can adjust clearmake's level of compatibility with other make programs:
- Suppressingclearmake's special features— Command options can selectively turn off such features as wink-in,comparison of build scripts, comparison of automatically-detecteddependencies, and creation of DOs and CRs. Configuration lookup can beturned off altogether, so that the standard timestamp-based algorithmis used for build avoidance.
- Enabling features of othermakeprograms — clearmake has several compatibility modes, which provide for partial emulations of popular make programs, such as Gnu Make and Sun Make.
Users can achieve absolute compatibility with other make programs by actually using those programs to perform builds. But in a build with a “standard” make,there is no build auditing, no configuration lookup, and no sharing ofDOs. The MVFS files created by the build are simply view-private files,not derived objects (unless the make program is executed in a clearaudit shell).
clearmake includes support for parallel building (concurrent execution of a set of build scripts), and for distributedbuilding (use of other hosts to execute build scripts). A commandoption specifies the number of hosts to use; hostnames to use are readfrom a build hosts file (Figure 5-5).Figure 5-5. Parallel and Distributed Building
For example, a user might perform a three-way build, all of whoseprocesses execute on a single multiprocessor compute server; anovernight build might be distributed across all the workstations in thelocal network.
Before starting a parallel build, clearmakeconsiders each target, determining whether an existing DO can bereused, or an actual build is required. Then, it organizes all theactual build work into a hierarchical master schedule.
clearmake then dispatches thebuild scripts to some or all the hosts listed in the build hosts file.It uses a dynamic load-balancing algorithm that makes best use of theavailable hosts, preferring lightly-loaded hosts to ones that arerelatively busy. Access to a “build server” host can be restricted toparticular times, to particular users working on particular remotehosts, and so on.
Execution of build scripts is managed on each remote host by the ClearCaseaudited build executor (abe), which is invoked through standard remote-shell facilities.
Many organizations develop multiple variants of their products,targeted at different platforms. But ClearCase may not currently beavailable for some of the platforms. How, then, can users build all therequired product variants?clearmake then dispatches thebuild scripts to some or all the hosts listed in the build hosts file.It uses a dynamic load-balancing algorithm that makes best use of theavailable hosts, preferring lightly-loaded hosts to ones that arerelatively busy. Access to a “build server” host can be restricted toparticular times, to particular users working on particular remotehosts, and so on.
Execution of build scripts is managed on each remote host by the ClearCaseaudited build executor (abe), which is invoked through standard remote-shell facilities.
One solution is cross-development— for example, perform a build on a SunOS host that producesexecutables for the unsupported host. ClearCase provides anothersolution — hosts on which ClearCase is not installed can still accessClearCase data, using standard network file-sharing services (such asNFS). This capability is termed non-ClearCase access.
Non-ClearCase access takes advantage of view transparency.Through automatic version-selection, a view makes any VOB appear to bea standard directory tree. Any such “VOB image” can be “exported” to anon-ClearCase host (Figure 5-6).
Figure 5-6. Non-ClearCase Access
Users on the non-ClearCase host can perform builds within these “VOB image” directory trees, using native makeutilities or other local build tools. The object modules andexecutables produced by such builds are stored in the view thatinstantiates the VOB image(s).
Since ClearCase software is not running on a non-ClearCase host,non-ClearCase access has some limitations. In many cases, there aresimple workarounds to the limitations.
- ClearCase build auditing, configuration lookup, and wink-in are not performed during builds on such hosts.
- Files produced by such builds are not derived objects, and haveno configuration records. (Users can employ remote-shell techniques toovercome this limitation, so that the files built on a non-ClearCasehost do become derived objects.)
- Users cannot execute ClearCase development commands, such as checkout and checkin, on non-ClearCase hosts. (But they can probably remote-login to a ClearCase host for such purposes.)
- Users cannot change the VOB image, by reconfiguring the view,from non-ClearCase hosts. (Again, a remote-login capability addressesthis issue.)
A development group can use DO versions to make its “product” (forexample, a library) available to other groups. Typically, the groupestablishes a “release area” in a separate VOB. For example:
- A library is built by its development group in one location — perhaps /vobs/monet/lib/libmonet.a.
- The group periodically “releases” the library by creating a new version of a publicly-accessible element — perhaps /vobs/publib/libmonet.a.
- When an executable orother file is ready to be released, a release engineer checks it in asa version of an element in the release tree.
- An appropriate version label (for example, REL2.4.3) is attached to that version, either manually by the engineer or automatically with a trigger.
- When all files to be shipped have been released in this way, a release engineer configures a view to select only versions with that version label. As seen through this view, the release tree contains exactly the set of files to be released.
- To cut a release tape, the engineer issues a command to copy the appropriately configured release tree.
Prev | Table of Contents | Next |
Chapter 4. ClearCase Views | Chapter 6. ClearCase Process Control |
We often have to write code using permissive programming languages like C and C++. They tend to generate hard-to-debug problems that can crash your applications. Thankfully, many compilers offer “sanitizers”. I discussed them in my post No more leaks with sanitize flags in gcc and clang. I strongly encourage the use of sanitizers as I think it is the modern way to write C and C++. When many people describe how impossibly difficult it is to build good software in C and C++, they often think about old-school bare metal C and C++ where the code do all sorts of mysterious things without any protection. Then they feel compelled to run their code in a debugger and to manually run through it. You should not write code this way! Get some tools! Sanitizers can catch undefined behaviour, memory leaks, buffer overflows, data races, and so forth.
Sanitizers are a game changer, they bring C++ closer to languages like Java and Rust. They do not bring the safety to production, since you probably do not want to use sanitizers in production or release, but as you are building and testing your code, they help you a great deal catch potential issues right away.
A competitive solution that people often use is a great tool called “valgrind“. It is a general-purpose tool that checks your software as it runs. Under Windows, you have related programs like Application Verifier and WinDbg.
I believe you should almost always use sanitizers when they are available. Here is a comparison between tools like valgrind and sanitizers.
- With the caveat that valgrind needs support for all the instructions your software is using, valgrind can run pretty much any software, even when you do not have the source code. Sanitizers work at the compiler level, so you need the source code. Thus if you need to debug a closed source library, sanitizers are unhelpful.
- Sanitizers can catch problems that valgrind will not catch. For example, it will catch undesirable undefined behaviour: code that may work right now but may not work if you use a different compiler or a different processor. They can catch unsafe memory accesses that will look safe to valgrind.
- Sanitizers are more precise. You often can turn on or off specific sanitizers for specific functions.
- If your compiler has sanitizers, you can run your tests with the sanitizers on simply by turning on some flags.
- Valgrind is slow. Like debuggers, it often does not scale. If you are working over large data sets, it might take a really long time. People often dismiss “execution time”, and it is easy to do if you work on toy problems, but performance is an essential quality-of-life attribute. I do not think you can run valgrind in a simulated production setting. However, you can compile your code with sanitizers and emulate a production setting. Sure, your throughput is going to be impacted, but the effect is not large. Code with sanitizers is not 10x slower, valgrind is.
- Sanitizers are relatively new and so the support is sometimes missing.
- For example, under macOS, Apple does not yet ship a compiler that can detect memory leaks, you need to install your own compiler.
- Even if you compile your code with debug symbols, it is common for the sanitizers to report the errors without proper links to the source code, you often need to fiddle with the system configuration.
- Under Linux, when using GNU GCC, I have found it necessary to use the gold linker to get good results (-fuse-ld=gold): the default link frequently gives me errors when I try to use sanitizers.
- The “memory sanitizer” that check that you do not read from uninitialized inputs is not available under GNU GCC and under LLVM requires you to manually replace the C++ standard library and possibly recompile all of your software with the sanitizer enabled (including all dependencies) if you want to avoid false positives.
- And Visual Studio has some of its own sanitizers, but it is largely behind LLVM. Better sanitizers may be coming to Visual Studio 2019.
- Furthermore, you cannot freely use all possible sanitizers at once.
So, sadly, there are cases when sanitizers are just not available to you. Yet I think it is a safe bet that all competitive C/C++ compilers will soon have powerful sanitizers.