Advice needed for a fast PCI->SCSI system

Hello Per,

You're in trouble if you have route the captured video through the ISA
boards and simultaneously write to disk. The ISA bus will become a major
bottleneck. Are these DMA bus masters??

When looking at you target system I'd include at least double the memory.
That way you can capture the entire 200-300MB into RAM, and then flush it to
disk. If you have only 256MB shared between the OS and your capture buffer,
you'll have lots of memory swap activity overhead. Memory is cheap, get 1GB.

Why are you including a 120GB IDE disk? Just for the OS? You can probably
use a lot smaller and save some money to get the extra memory. If you insist
on using a 120GB disk, you may want to allocate only a small partition (8 GB
or so) on it for the OS, and leave the rest unpartitioned. You'll be amazed
how fast your disk will be, as it only has to seek across a very small slice
of the disk. Also, being on the outer part of the disk means you get the
best transfer rate.

Software RAID is OK if your CPU isn't supposed to do anything more useful at
the same time. In your situation with I/O bound processing it's OK. In
situations where the CPU is actively participating in some process it may be
better to offload RAID to hardware and have the CPU do what it's intended
for; process user data.

Rob

Rob Turk wrote:
>

I agree with expanding the memory considerably, but NOT because the OS
will let you write the whole load to RAM and then flush to hard disk.
As far as all my testing has shown, Windows 95 and later all have
diabolical memory management.

What might work better is a cache controller. I used to use them in all
my high-end systems in the early 90s. I don't believe anyone does them
now, so you would have to consider something like a very high-end RAID
card that will accept at least 256MB of cache memory. And don't get
Adaptec - word on the street is that they are crap. (I have a 2100S
with 128MB cache memory and 2 x 10K SCSI drives in RAID 0; this setup
performs slower than a single IDE drive running of the motherboard's
controllers.) Suggest you look at Mylex or 3ware (or 4ware - whatever.)

Odie

Nuz

I agree to the fact that RAM is cheap, but I don't see how it would
speed up the processing of downloading my buffered data from the
frame-grabber onto disk. Keep in mind this is not an "ordinary"
frame-grabber. The camera connected to it captures frames at up to
8000 frames per second. All frames captured are saved on the PCI
board, which can amount to 300 Megs of data. After the grabbing, the
data is downloaded to disk in one go (not frame per frame as they are
being grabbed from the camera).

In any case, my system/memory BUS is much faster that my PCI bus,
right? So why would more RAM speed up things here?

Are you suggesting first copying the data from the PCI card's buffer
into RAM, and from there onto disk? One vendor we dealt with suggested
we use flash RAM, but all this does is to create an additional process
that will take up more time. In our case, the reason we need to speed
up the download time from the PCI card is that we have an experiment
running that will sit idle, waiting for the PC to finish off the
save-job, and then move on. We want to reduce this waiting time.

Rob, the reason we're using a large IDE drive is that we need to save
the data from the SCSI drives onto a slower medium later. We may
capture up to 60 GB of video-data in one day, but we don't want spend
all our money on expensive SCSI drives. Between our experiments, when
the need for speed is not all that important, we will copy all our
files over to the IDE drive, leaving space on the SCSI's to capture
another experiment's worth of video data. Hope that makes sense to
you.

Per

I'd use a dual CPU system -- the 2nd CPU can do wonders
under heavy load, which will be your case

None as far as I'm aware of. This is an industrial PC, with a 13 slot
backplate (4 ISA, 7 PCI, 2 CPU). The CPU board we consider using is a
Rocky 4784EV, Pentium 4 2 GHz.

Any idea how this will affect performance? Will the ISA slow down the
system?

Per

Currently, I'm saving from the PCI card to a 7.2k RPM IDE drive. The
CPU is more or less idle during this operation. I don't think the
process of saving will be any more CPU intensive with SCSI drives
(probably less), even with a software controlled RAID 0 system, as
the striping process takes little effort to deal with for a fast CPU.
Indeed, we have a 3x36GByte RAID 5 system on another PC with similar
specs as the new one we wish to purchase. Write access on this PC is
about ten faster than our current IDE drive (Sandra benchmark).

The main bottleneck in my proposed system is the write access to the
harddrives, thus the suggested RAID 0 system (which in theory should
be even faster than our RAID 5 system).

Can you enlighten me further as to why you think more CPU power will
speed up saving from a PCI buffer to disk?

To stay in tune with my first post, however, one my questions was the
effect of the ISA bus on the overall performance on the system. See my
previous post for specs on the proposed system. The ISA cards may,
apparently, slow down things if the ISA cards are being used. I am
uncertain, however, exactly what that means. During the PCI->disk save
operation, for instance, neither ISA card will do anything. Will they
affect performance simply by being connected to the rest of the
system?

Also, does anyone have experience with industrial versus desktop PCs?
I have been told that the latter generally outperform industrial PCs.
Does this have to do something with the fact that desktop PCs have the
CPU and the slots integrated on the same motherboard (versus a
separate backplate and CPU board in the case of an industrial PC).

Per

--
Mike Walsh
West Palm Beach, Florida, U.S.A.

The answer has been staring me in the face all along.

I use a program called RAMDisk Pro.

This allows you to create a RAM disk (obviously you need a fair amount
of system RAM) and assign it a drive letter. So, if you create your RAM
disk with 512MB of system memory, and call it Z: you will be able to
treat it just like a normal drive. However, it writes at 300MB to 400MB
(megabytes - not bits) per second.

The disadvantage is that you lose everything when you reboot your
system.

If you can't find a copy, let me know and I'll send you mine (needs
registration if you decide to keep it.)

I use it as a scratch disk for Adobe products - it is seriously quick.

Odie

Do you copy data from the PCI memory to system RAM then write it with an
unbuffered I/O of 64KB or more?

The disk write in this situation should be close to 100 MB/s with NT's RAID 0.
What rates do you get reading the RAM on the PCI card? Can you write from PCI
memory directly?

The mere presence of an ISA bridge can slow down CPU initiated I/O to PCI
space by one cycle. It does not affect PCI bus mastering.

They will have an adverse effect on system performance if they use ISA
DMA busmastering. This is because busmaster transfers over the ISA-PCI
bridge are very inefficient.

If the ISA cards transfer data using PIO, they will probably be OK.

No.

--
Security-wise, NT is a server with a 'Kick me'
sign taped to it. -- Peter Gutmann