มีใครแก้ไข PTLOS แบบฝึกหัด 4.1 ได้บ้างไหม?


19

นี้การออกกำลังกายที่ได้รับในทฤษฎีความน่าจะเป็น: ตรรกะของวิทยาศาสตร์โดยเอ็ดวินเจย์นส์, ปี 2003 มีวิธีการแก้ปัญหาบางส่วนเป็นที่นี่ ฉันได้หาทางแก้ปัญหาบางส่วนที่กว้างขึ้นและสงสัยว่ามีคนอื่นแก้ไขมันได้ไหม ฉันจะรอสักครู่ก่อนโพสต์คำตอบของฉันเพื่อให้ผู้อื่นได้ไป

เอาล่ะสมมติว่าเรามีnพิเศษร่วมกันและสมมติฐานหมดจดแสดงโดยHi(i=1,,n) ) ต่อไปสมมติว่าเรามีชุดข้อมูลmแสดงโดยDj(j=1,,m) ) อัตราส่วนความน่าจะเป็นสำหรับข้อสมมติฐานที่ i ถูกกำหนดโดย:

LR(Hi)=P(D1D2,Dm|Hi)P(D1D2,Dm|H¯i)

โปรดทราบว่าสิ่งเหล่านี้เป็นความน่าจะเป็นตามเงื่อนไข ตอนนี้สมมติว่าได้รับ ith สมมติฐานเมตรชุดข้อมูลมีความเป็นอิสระเพื่อให้เรามี:Him

P(D1D2,Dm|Hi)=j=1mP(Dj|Hi)(i=1,,n)Condition 1

ตอนนี้มันจะค่อนข้างสะดวกถ้าตัวหารยังรวมอยู่ในสถานการณ์นี้ด้วยดังนั้นเราจึงมี:

P(D1D2,Dm|H¯i)=j=1mP(Dj|H¯i)(i=1,,n)Condition 2

สำหรับในกรณีนี้อัตราส่วนความน่าจะเป็นแยกเป็นผลิตภัณฑ์ที่มีขนาดเล็กลงสำหรับแต่ละชุดข้อมูลดังนั้นเราจึงมี:

LR(Hi)=j=1mP(Dj|Hi)P(Dj|H¯i)

ดังนั้นในกรณีนี้แต่ละชุดข้อมูล "จะออกเสียงลงคะแนนสำหรับHi " หรือ "โหวตกับHi " เป็นอิสระจากชุดข้อมูลอื่น ๆ

แบบฝึกหัดคือการพิสูจน์ว่าถ้าn>2 (มากกว่าสองข้อสมมุติ) ไม่มีวิธีที่ไม่น่าสนใจเช่นนี้ที่แฟคตอริ่งสามารถเกิดขึ้นได้ นั่นคือถ้าคุณสมมติว่าเงื่อนไข 1 และเงื่อนไข 2 ค้างไว้ปัจจัยส่วนใหญ่: แตกต่างจาก 1 และดังนั้นชุดข้อมูลเพียง 1 ชุดเท่านั้นที่จะมีส่วนในอัตราส่วนความน่าจะเป็น

P(D1|Hi)P(D1|H¯i)P(D2|Hi)P(D2|H¯i)P(Dm|Hi)P(Dm|H¯i)

ฉันเองพบว่าผลลัพธ์นี้น่าสนใจมากเพราะโดยทั่วไปแล้วมันแสดงให้เห็นว่าการทดสอบสมมติฐานหลายรายการนั้นไม่มีอะไรนอกจากชุดการทดสอบสมมติฐานคู่


ฉันสับสนเล็กน้อยเกี่ยวกับดัชนีใน ; เป็นˉ Hฉัน = หาเรื่องสูงสุดชั่วโมงH ฉัน P ( D 1 , ... D เมตร| H ) ? หรือมันคือˉ H i = arg max h { H 1 , , H n } P ( D 1 , D m | h )H¯iH¯i=argmaxhHiP(D1,Dm|h)H¯i=argmaxh{H1,,Hn}P(D1,Dm|h)? Seems like it ought to be the latter, but then I'm not sure why the subscript. Or maybe I'm missing something else entirely :)
JMS

@JMS - หมายถึงคำสั่งตรรกะ " H ฉันเป็นเท็จ" หรือว่าหนึ่งในสมมติฐานอื่น ๆ เป็นจริง ดังนั้นใน "พีชคณิตแบบบูล" เรามี¯ HฉันH 1 + H 2 + + H ฉัน- 1 + H ฉัน+ 1 + + H n (เพราะสมมติฐานที่มีความพิเศษและครบถ้วนสมบูรณ์)H¯iHiH¯iH1+H2++Hi1+Hi+1++Hn
probabilityislogic

I feel like there has to be a more intuitive solution than the algebra given in Sanders' partial solution. If the data are independent given each of the hypotheses then this continues to hold when the priors of the hypothesis are varied. And somehow, the result is that the same must apply for the conclusion...
charles.y.zheng

@charles - I know exactly how you feel. I thought I could derive it using some qualitative inconsistency (Reductio ad absurdum), but I couldn't do it. I Could extend Sander's maths though. And it is Condition 2 which is "the dodgy one" in terms of what the result means.
probabilityislogic

@probabilityislogic "it basically shows that multiple hypothesis testing is nothing but a series of binary hypothesis tests." Please, could you expand on this sentence? By reading page 98 from Jaynes' book, I understand that you can reduce testing of H1,,Hn to testing H1 against each other hypothesis and then somehow normalize to get the posterior for H1, but I do not understand why this would follow from the results of Excercise 4.1.
Martin Drozdik

คำตอบ:


7

The reason we accepted eq. 4.28 (in the book, your condition 1) was that we assumed the probability of the data given a certain hypothesis Ha and background information X is independent, in other words for any Di and Dj with ij:

P(Di|DjHaX)=P(Di|HaX)(1)
Nonextensibility beyond the binary case can therefore be discussed like this: If we assume eq.1 to be true, is eq.2 also true?

P(Di|DjHa¯X)=?P(Di|Ha¯X)(2)
First lets look at the left side of eq.2, using the multiplication rule:

P(Di|DjHa¯X)=P(DiDjHa¯|X)P(DjHa¯|X)(3)
Since the n hypotheses {H1Hn} are assumed mutually exclusive and exhaustive, we can write:
Ha¯=baHb
So eq.3 becomes:
P(Di|DjHa¯X)=baP(Di|DjHbX)P(DjHb|X)baP(DjHb|X)=baP(Di|HbX)P(DjHb|X)baP(DjHb|X)
ba), the equal terms in the nominator and denominator, P(DjHb|X), cancel out and eq.2 is proved correct, since Hb=Ha¯. Therefore equation 4.29 can be derived from equation 4.28 in the book. But when we have more than two hypotheses, this doesn't happen, for example, if we have three hypotheses: {H1,H2,H3}, the equation above becomes:
P(Di|DjH1¯X)=P(Di|H2X)P(DjH2|X)+P(Di|H3X)P(DjH3|X)P(DjH2|X)+P(DjH3|X)
In other words:
P(Di|DjH1¯X)=P(Di|H2X)1+P(DjH3|X)P(DjH2|X)+P(Di|H3X)1+P(DjH2|X)P(DjH3|X)
The only way this equation can yield eq.2 is that both denominators equal 1, i.e. both fractions in the denominators must equal zero. But that is impossible.

1
I think the fourth equation is incorrect. We should have P(DiDjHb|X)=P(DiHB|X)P(Dj|HbX)
probabilityislogic

Thank you very much probabilityislogic, I was able to correct the solution. What do you think now?
astroboy

I just don't understand how Jaynes says: "Those who fail to distinguish between logical independence and causal independence would suppose that (4.29) is always valid".
astroboy

I think I found the answer to my last comment: right after the sentence above Jaynes says: "provided only that no Di exerts a physical influence on any other Dj". So essentially Jaynes is saying that even if they don't have physical influence, there is a logical limitation that doesn't allow the generalization to more than two hypotheses.
astroboy

After reading the text again I feel my last comment was not a good answer. As I understand it now, Jayne's wanted to say: "Those who fail to distinguish between logical independence and causal independence" would argue that Di and Dj are assumed to have no physical influence. Thus they have causal independence which for them implies logical independence over any set of hypotheses. So they find all this discussion meaningless and simply proceed to generalize the binary case.
astroboy

1

Okay, so rather than go and re-derive Saunder's equation (5), I will just state it here. Condition 1 and 2 imply the following equality:

j=1m(kihkdjk)=(kihk)m1(kihkj=1mdjk)
where
djk=P(Dj|Hk,I)hk=P(Hk|I)

Now we can specialise to the case m=2 (two data sets) by taking D1(1)D1 and relabeling D2(1)D2D3Dm. Note that these two data sets still satisfy conditions 1 and 2, so the result above applies to them as well. Now expanding in the case m=2 we get:

(kihkd1k)(lihld2l)=(kihk)(lihld1ld2l)

kilihkhld1kd2l=kilihkhld1ld2l

kilihkhld2l(d1kd1l)=0(i=1,,n)

The term (d1ad1b) occurs twice in the above double summation, once when k=a and l=b, and once again when k=b and l=a. This will occur as long as a,bi. The coefficient of each term is given by d2b and d2a. Now because there are i of these equations, we can actually remove i from these equations. To illustrate, take i=1, now this means we have all conditions except where a=1,b=2 and b=1,a=2. Now take i=3, and we now can have these two conditions (note this assumes at least three hypothesis). So the equation can be re-written as:

l>khkhl(d2ld2k)(d1kd1l)=0

Now each of the hi terms must be greater than zero, for otherwise we are dealing with n1<n hypothesis, and the answer can be reformulated in terms of n1. So these can be removed from the above set of conditions:

l>k(d2ld2k)(d1kd1l)=0

Thus, there are n(n1)2 conditions that must be satisfied, and each conditions implies one of two "sub-conditions": that djk=djl for either j=1 or j=2 (but not necessarily both). Now we have a set of all of the unique pairs (k,l) for djk=djl. If we were to take n1 of these pairs for one of the j, then we would have all the numbers 1,,n in the set, and dj1=dj2==dj,n1=dj,n. This is because the first pair has 2 elements, and each additional pair brings at least one additional element to the set*

But note that because there are n(n1)2 conditions, we must choose at least the smallest integer greater than or equal to 12×n(n1)2=n(n1)4 for one of the j=1 or j=2. If n>4 then the number of terms chosen is greater than n1. If n=4 or n=3 then we must choose exactly n1 terms. This implies that dj1=dj2==dj,n1=dj,n. Only with two hypothesis (n=2) is where this does not occur. But from the last equation in Saunder's article this equality condition implies:

P(Dj|H¯i)=kidjkhkkihk=djikihkkihk=dji=P(Dj|Hi)

Thus, in the likelihood ratio we have:

P(D1(1)|Hi)P(D1(1)|H¯i)=P(D1|Hi)P(D1|H¯i)=1 ORP(D2(1)|Hi)P(D2(1)|H¯i)=P(D2D3,Dm|Hi)P(D2D3,Dm|H¯i)=1

To complete the proof, note that if the second condition holds, the result is already proved, and only one ratio can be different from 1. If the first condition holds, then we can repeat the above analysis by relabeling D1(2)D2 and D2(2)D3,Dm. Then we would have D1,D2 not contributing, or D2 being the only contributor. We would then have a third relabeling when D1D2 not contributing holds, and so on. Thus, only one data set can contribute to the likelihood ratio when condition 1 and condition 2 hold, and there are more than two hypothesis.

*NOTE: An additional pair might bring no new terms, but this would be offset by a pair which brought 2 new terms. e.g. take dj1=dj2 as first[+2], dj1=dj3 [+1] and dj2=dj3 [+0], but next term must have djk=djl for both k,l(1,2,3). This will add two terms [+2]. If n=4 then we don't need to choose any more, but for the "other" j we must choose the 3 pairs which are not (1,2),(2,3),(1,3). These are (1,4),(2,4),(3,4) and thus the equality holds, because all numbers (1,2,3,4) are in the set.


I am beginning to doubt the accuracy of this proof. The result in Saunders maths implies only n non linear constraints on the djk. This makes djk only have n degrees of freedom instead of 2n. However to get to the n(n1)2 conditions a different argument is required.
probabilityislogic

0

For the record, here is a somewhat more extensive proof. It also contains some background information. Maybe this is helpful for others studying the topic.

The main idea of the proof is to show that Jaynes' conditions 1 and 2 imply that

P(Dmk|HiX)=P(Dmk|X),
for all but one data set mk=1,,m. It then shows that for all these data sets, we also have
P(Dmk|H¯iX)=P(Dmk|X).
Thus we have for all but one data set,
P(Dmk|HiX)P(Dmk|H¯iX)=P(Dmk|X)P(Dmk|X)=1.
The reason that I wanted to include the proof here is that some of the steps involved are not at all obvious, and one needs to take care not to use anything else than conditions 1 and 2 and the product rule (as many of the other proofs implicitly do). The link above includes all these steps in detail. It is on my Google Drive and I will make sure it stays accessible.


Welcome to Cross Validated. Thank you for your answer. Can you please edit you answer to expand it, in order to include the main points of the link you provide? It will be more helpful both for people searching in this site and in case the link breaks. By the way, take the opportunity to take the Tour, if you haven't done it already. See also some tips on How to Answer, on formatting help and on writing down equations using LaTeX / MathJax.
Ertxiem - reinstate Monica

Thanks for your comment. I edited the post and sketched the main steps of the proof.
dennis
โดยการใช้ไซต์ของเรา หมายความว่าคุณได้อ่านและทำความเข้าใจนโยบายคุกกี้และนโยบายความเป็นส่วนตัวของเราแล้ว
Licensed under cc by-sa 3.0 with attribution required.