GENERAL
PROTOCOL It is paramount to have a set of basic rules to test equipment and compare components, to follow those rules in order to stablish a sistematic and rigorous protocol upon wich the verification and validity of a hypothesis is structured. The main axis is the blind test. This is the only method that allows to obtain real results about the existance or not of differences when comparing 2 or more HiFi components..
1) Surrounding Issues Control
Adjust, match, regulate and supervise any and all kind of variables that may affect the normal development of the test and that may interfere with the results.
For digital sources it is fundamental and mandatory to match output levels given the fact that if a CDP has a higher output level than the other it may lead to believe that it sounds differently when in reality all what’s happening is that it’s sounding louder.
Any kind of involuntary manipulation or error of fundamental surrounding issues that may distort the final result of the test will generate a repetition of the test.
Any suspicion about intentionally manipulation with the sole intention of sabotage the test will immediatly call for the cancellation of all proceedings.CDP--------> Match output levels (UNAVOIDABLE)
a) Can be done via the preamp individual input gain control (if available)
b) Can be done using one of the CDP output level control (if available)
c) Can be done using an external level matching unit2) Pilot test.
Let’s assume a test on a Hifi unit “A” and another HiFi unit “B”. Initially, the tester (subject) will be allowed a period of acclimatization, that is, listening to each component knowing at all times what he’s listening to. The pilot testing will be designed to be simple, that is, with up to several trials in between A & B but always making sure that the tester is knowledgeable of what is sounding and what he’s listening to at all times.
a) Questionary: “Do you hear differences in between A & B?”
In the event that the subject (tester), knowing at all times what he’s listening to (A or B), replies NO, automatically the testing procedures are finished with the statement:
THERE ARE NO DIFFERENCESIf there are multiple subjects performing the pilot test, all of them must conclude that there were heard differences to declare the pilot test finished, and the statement will be: YES, THERE ARE DIFFERENCES.
b) Blind test
If the subject replies YES to the question “do you hear differences in between A&B?”, then we proceed to the actual blind testing procedures, that is, without knowing what component is now sounding.
b.1) A minimum of 10 trials are conducted
b.2) The sound samples will be of minimum 10 sec. duration
b.3) It is considered that there are differences when there is a 90% accurate replies
b.4) When accuracy is in between 80% and 90%, the test is repeated
b.5) Accurate replies below 80% is considered randomness or chance (lucky draw)
Before initiating the test, a matrix or master file is created based on aleatory trials that will be launched to the subject (tester).
Flipping a coin will generate the secuency of trials to be launched and this secuency will be written on the matrix form. That secuency (generated by the coin) is the frecuency exposed to the subject.1) Continuous Sequence (CS)
A trial is equivalent to one sound sample for each component, being A or B, using an “x” to check the form designed to that effect. I.e. let’s say that the first trial indicates that it’d sound component A, the subject (tester) replies if he thinks it’s A or B, then, it’ll sound component B and the subject again must reply if A or B is sounding, and so on.
The reply “x” must then be “X” to A or B
Example:
CS = A(x) B(x) A(x) A(x) B(x) B(x) A(x) B(x) B(x) A(x)
2) Binary Sequence (BiS)
One trial is equivalent to 2 sound samples of the components, that may be A or B, being x the subject (tester) reply on the form designed to that effect. That is, first one component is sounding, then the oteher and right after the tester replies if he thinks it’s A or B. Lauching groups of 2 sound samples generates a reply on each pair, and so on.
The reply “x” must then be------- to AB or BA
Example:
BiS= AB(x) AB(x) BA(x) BA(x) AB(x) BA(x) BA(x) AB(x) AB(x) BA(x)
3) Mixed Sequence (MS)It is basically the same as BiS only adding the variable that on a same trial there might be included the same component, that is, repeating sound source.
The reply “x” must the be----------------to AA or BB or AB or BA
Example:
MS: AA(x) AB(x) BA(x) BB(x) BB(x) AA(x) BA(x) AA(x) AB(x) BB(x)
1) L.A.N.C.I.S. (Listening Audio Nodes and Concurrent Immediate Sequences)
This method is designed by the authors in order to diminish to the minimum the potential interferences and pernicious effects of time to audible memory. One of the greatest mistakes when listening and comparing audio equipments has been relying on audible memory. The Lancis method consists in comparing components by means of trials where the signal is commuted from one to the other component while both are using the same source material and are sinchronized to each other in time. This allows to compare instantly both components and detect (if any) differences since switching to one from another is made almost instantly.
Example:
We want to know if 2 CD players sound different to each other and we play on both 2 original recordings (same label, same batch) of any given author and track (chosen by the tester). We synchronize both CDP to sound exactly at the same time. We simply commute from the preamp from one to the other and compare instantly and unequivocally both players.2) C.O. B.O. (Commutations Of Blindtest Occurrences)
It is first heard a complete track on one component and then the same track is heard on the 2nd component. This is a less precise method since in between one trial and the other, the time elapsed makes audible memory necessary and it may distort the results. It makes it harder to conduct the test successfully even if there were differencies, the perception threshold is challenged to its limits and may provoque mental confusion.
Selecting a method:
It is reccomended to use the LANCIS Method since it allows more presition when having to make a desition about if there are or not differencies. To explain this better, we use this simple example:
Imagine having 2 paper sheet with a drawing of a cat on each, but with 7 small differencies in between one and the other drawing: How’s easily detected those 7 differencies? Having both sheet next to each other and look at both comparing instantly the drawings or perhaps fixing the look for a while on one sheet, memorize it, hide it away and only then look to the next sheet trying to recall or spot the differencies?
MATRIX-HIFI
HOMOLOGATION PROCESS
To consider an experiment valid and reliable it must be conducted following the rules of the Official Matrix HiFi General Protocol.
Every trial, test or findings as a result of, must be, prior to be accepted as valid and published on the web site, submitted to the analysis of the Matrix HiFi Evaluation Comitee to verify that the significant variables were under control and the method was followed rigourosly.
The Matrix HiFi Evaluation Comitee is composed by sound technicians, hifi proffessionals, electronic and data engineer and, enthusiasts with vast experience in blind testing and evaluation of high end systems. Said comitee members are the ones to evaluate and consider the validity, rigourousity and fiability of any experiment conducted by any person foreign to MatrixHiFi.
